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Executive  Summary 

SECTION  1 :  INTRODUCTION  AND  TERMS  OF  REFERENCE  OF  THE  STUDY 

Health  services  internationally  are  emphasizing  the  need  to  ensure  that  medicine  and  health 
care  delivery  are  evidence-based.  For  those  involved  with  health  related  research,  this 
development  focuses  attention  on  the  importance  of  a  strong  research  foundation  and  it 
enhances  the  potential  contribution  that  relevant  research  can  make.  Equally,  it  places  an  onus 
on  the  research  community  to  provide  evidence  of  the  value  of  its  own  activities,  and  it  is  one  of 
the  factors  leading  to  a  growing  interest  in  methods  for  assessing,  either  retrospectively  or 
prospectively,  the  value  of  research  undertaken  or  proposed. 

A  small  team  of  researchers,  at  the  Health  Economics  Research  Group  (HERG)  at  Brunei 
University,  have  been  considering  how  best  the  'payback'  or  benefits  of  funded  research 
projects  might  be  conceptualized,  described  and  measured.  This  work,  initially  for  the  UK 
Department  of  Health,  proposed  a  conceptual  categorisation  of  the  benefits  arising  from  health 
services  research  and  introduced  a  model  of  'payback'.  These  are  referred  to  as  the 
Buxton/Hanney  payback  categories  and  the  Buxton/Hanney  payback  model,  and  they  are 
summarized  in  Figures  1  and  2.  They  have  been  applied,  by  the  HERG  Team,  in  a  number  of 
ex  post  case  studies  of  applied  health  services  research  projects  in  the  UK. 

There  has  been  a  considerable  interest  in  the  wider  applicability  of  this  work  both  to  HSR 
undertaken  in  other  countries,  and  to  basic  science  or  clinical  research  that  may  not  yet  be 
immediately  applicable  to  health  policy  or  practice.  Professor  Martin  Buxton  was  invited  by  the 
Alberta  Heritage  Foundation  for  Medical  Research  to  test  and  develop  the  Buxton/Hanney 
model  in  the  context  of  its  own  programs  of  research.  The  objective  was  not  to  assess  the 
overall  payback  from  AHFMR-funded  research,  nor  to  'audit'  individual  projects.  Rather,  the 
work  focused  on  testing  the  approach  to  see  if  it  might  prove  useful  to  the  AHFMR  in  reviewing 
the  payback  achieved,  or  achievable,  from  the  research  it  funds.  This  involved  both  confirming 
that  the  approach  was  transferable  from  the  UK  context,  and  the  additional  challenge  of 
establishing  whether  the  approach,  or  an  adaptation  of  it,  could  be  applied  to  other  areas  of 
health  research  such  as  basic  biomedical  and  clinical  science. 
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SECTION  2:  METHODS 

A  selective  case  study  approach  was  adopted  in  which  the  studies  would  be  developed  only  as 
far  as  was  necessary  to  begin  to  test  the  applicability  of  the  model.  This  was  consistent  with  the 
objective  to  test  the  approach,  rather  than  the  research,  or  the  researchers,  chosen  as  test 
cases. 

In  order  to  test  whether  the  Buxton/Hanney  approach  could  readily  be  applied  to  HSR  in  this 
different  context,  two  case  studies  were  chosen  that  represented  good  examples  of  applied 
HSR  funded  by  the  AHFMR.  Interviews  with  the  researchers  were  used  to  verify  conclusions 
drawn  from  existing  written  materials  and  to  identify  any  payback  in  other  categories  that  might 
have  been  missed.  In  addition,  two  case  studies  were  undertaken  on  Health  Technology 
Assessment  (HTA)  projects  from  the  AHFMR  HTA  program. 

To  test  the  model  or  a  variant  of  it,  on  both  clinical  research  and  basic  biomedical  research,  the 
work  of  four  distinguished  researchers,  who  had  all  received  extensive  AHFMR  funding  over  a 
number  of  years,  was  chosen.  In  each  case,  the  mode  of  working  was  to  undertake  a  desk 
review  of  materials  on  file  followed  by  interviews  with  the  researchers  themselves. 

SECTION  3:  IS  THE  WORK  ON  METHODS  FOR  ASSESSMENT  OF  RESEARCH  BENEFITS 
INTERNATIONALLY  GENERALISABLE,  OR  IS  AHFMR  OR  ITS  CONTEXT  DIFFERENT? 

The  Buxton/Hanney  model  was  developed  specifically  in  the  UK  context,  but  drew  on  lessons 
from  studies  undertaken  elsewhere,  including  work  undertaken  in  Canada.  Given  the  general 
concern  that  research  findings  are  not  necessarily  geographically  generalizable  from  one 
country  or  setting  to  another,  it  was  a  necessary  first  step  to  test  whether  the  Buxton/Hanney 
approach,  would  appear  to  be  valid  and  appropriate  in  this  different  context. 

The  key  characteristics  of  the  local  context  were  examined,  focusing  mainly  on  an  analysis  of 
the  explicit  and  implicit  objectives  of  health  research  in  Alberta.  While  there  are  many 
similarities  to  the  UK  context,  there  are  a  number  of  ways  in  which  the  work  of  the  AHFMR  does 
differ  from  that  in  the  UK  for  which  the  model  was  originally  created. 

A  brief  look  at  the  characteristics  and  context  of  health  care  systems  in  the  UK  and  Alberta 
suggest  that  there  are  many  relevant  similarities,  including  the  importance  of  public  provision 
and  financing  of  much  health  care,  and  concerns  about  increasing  the  effectiveness  and 
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efficiency  of  ineaith  care  provision.  Similar  public,  political  and  professional  pressures  provide 
the  context  to  research.  Specifically,  the  debate  about  the  need  to  justify  spending  on  research 
appears  to  be  very  similar  in  Alberta  as  in  the  UK  and  elsewhere. 

If  performance  is  to  be  evaluated  it  is  essential  that  the  evaluation  criteria  reflect  as  closely  as 
possible  the  objectives  of  the  activity  in  question.  The  various  statements  of  mission,  goals  and 
objectives  of  the  AHFMR,  the  Alberta  Health  Ministry,  and  the  Alberta  Science  and  Research 
Authority  are  in  no  way  exceptional  and  the  Buxton/Hanney  categories  or  payback  are  quite 
consistent  with  them. 

However,  there  are  some  important  differences  in  emphasis  in  Alberta  that  need  to  be  reflected 
in  work  on  payback.  Most  of  the  development  of  substantial  research  interests  in  health 
sciences  in  Alberta  is  relatively  recent.  The  AHFMR  has  focussed  on  developing  a  capacity  and 
building  an  infrastructure  for  health  research.  This  implies  an  emphasis  on  payback  in  terms  of 
'development  of  research  skills,  personnel  and  research  capacity'.  Much  of  the  early  funding 
has  aimed  to  build  research  teams,  to  encourage  and  develop  younger  research  staff  and  to 
equip  them  to  begin  to  be  competitive  at  a  national  and  international  level.  It  has  been 
recognized  that  the  health  paybacks  would  occur  in  the  longer  term,  from  subsequent  phases  in 
this  strategic  development.  It  has  also  meant  that  research  funding  has  been  seen  as  having  a 
broader  educational  role  in  helping  to  educate  future  researchers. 

Moreover  there  is  recognition,  within  the  Government  of  Alberta,  of  the  economic  importance  of 
developing  science  and  R&D  capabilities,  not  least  in  health  related  areas,  as  part  of  broad 
long-term  provincial  strategy.  This  suggests  that  the  category  of  broader  economic  benefits 
from  research  may  be  viewed  as  important  for  this  strategic  investment  as  the  benefits  to  the 
health  sector  itself. 

A  further  important  contextual  factor  is  that  the  AHFMR  funds  a  comprehensive  range  of 
activities,  including  fundamental  biology  through  clinical  investigation  to  studies  on  population 
health  in  the  community.  The  single  largest  component  of  funding  is  the  personnel  support 
programs,  which  contribute  to  the  salary  costs  of  faculty-level  investigators  in  universities  and 
affiliated  institutions.  In  addition,  it  provides  funding  for  training  programs,  major  equipment  and 
facilities,  visiting  lecturer  and  travel  awards,  visiting  professorships,  infrastructure  grants,  and 
conferences  and  workshops.  Further,  through  collaboration  with  Alberta  Health,  the  AHFMR 
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administers  a  Health  Research  Fund  for  individual  project  support  and  operates  a  Health 
Technology  Assessment  Unit. 

SECTION  4:  APPLYING  THE  BUXTON/HANNEY  MODEL  TO  AHFMR-FUNDED  HSR  - 
RESULTS  FROM  THE  CASE  STUDIES 

The  Buxton/Hanney  payback  model  was  tested  on  two  specific  areas  of  HSR  funded  by  the 
AHFMR,  and  these  two  case  studies  have  been  summarized  using  the  framework  of  the  model. 
The  case  studies  relate  to  the  evaluation  of  a  novel  diagnostic  and  treatment  method  for  sleep 
apnea,  and  to  the  value  of  lumbar  spine  radiographs  in  the  management  of  low  back  pain.  As 
expected,  no  major  difficulties  were  encountered  in  applying  the  model  to  either  of  these  cases 
and  the  research  sequence  model  provided  a  useful  framework  for  reviewing,  ex  post,  what  had 
been  the  benefits  from  the  research. 

In  that  neither  of  the  streams  of  research  studied  had  been  commissioned,  an  aspect  of 
particular  interest  was  to  see  how  the  process  of  'research  needs  assessment'  had  occurred.  It 
was  possible  to  reconstruct  how  the  researchers  had  identified  the  ohginal  topics  and  how  they 
had  continued  to  identify  key  research  sub-topics  and  objectives  at  each  stage  of  their  work. 

The  other  two  HSR  case  studies  were  examples  of  health  technology  assessments  undertaken 
by  the  HTA  Unit  of  the  AHFMR,  relating  to  stereotactic  radiosurgery  and  telephone  nurse  triage 
services.  These  commissioned  studies  were  also  readily  amenable  to  analysis  using  the 
Buxton/Hanney  approach. 

Together  they  provide  clear  examples  that  suggest  that  the  approach  is  applicable  to  HSR  in  the 
Alberta  context.  They  also  show  that  an  analyst,  who  has  no  previous  experience  of  the  model 
can  use  the  approach,  and  that  it  is  possible  to  go  a  considerable  way  in  analyzing  payback 
from  a  project  using  existing  on-file  sources  of  information.  The  interviews  with  the  external 
researchers  tended  to  be  mainly  confirmatory,  rather  than  essential.  Certainly  it  would  be 
possible  to  undertake  such  analysis  routinely,  if  an  after-project  review  report  were  provided  by 
researchers  some  time,  say  two  years,  after  the  end  of  funding. 
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SECTION  5:  EXTENDING  THE  MODEL  TOWARDS  CLINICAL  AND  BASIC 
BIOMEDICAL  RESEARCH 

Previously,  the  Buxton/Hanney  approach  has  been  applied  principally  to  examples  of  applied 
health  services  research.  Where  the  research  has  been  methodological,  it  has  been  observed 
that  the  payback  will  typically  arise  through  the  subsequent  use  of  the  methods  in  applied 
research.  By  extension,  the  same  argument  might  be  applicable  to  more  basic  research  in 
health  sciences,  where  health  service  or  economic  benefits  (and  other  benefits)  arise  as  a  result 
of  the  incorporation  of  the  results  of  the  basic  biomedical  research  into  subsequent  more 
applied  research. 

The  scientific  literature  highlighted  studies  that  had  retrospectively  traced  the  path  from 
application  back  to  basic  science.  To  understand  the  eventual  payback  from  particular 
investments  in  basic  science,  it  would  be  necessary  to  trace  the  linkages  fonA/ard  from  a  single 
piece  or  program  of  research,  as  results  and  concepts  are  taken  up,  or  influence,  an 
increasingly  disparate  range  of  subsequent  research  and  eventual  application.  In  principle,  this 
forward  mapping  could  identify  complex  linkages  between  basic  research  and  future  health 
benefits.  Although  some  basic  research  ideas  may  be  taken  up  and  applied  rapidly,  others  may 
only  be  applied  after  a  significant  time  has  elapsed.  Because  basic  science  may  have  very 
disparate  and  unforeseeable  impacts,  attempts  to  identity  and  describe  them  may  involve  much 
wider  searches  than  might  typically  be  needed  to  follow-up  on  a  specific  piece  of  applied 
research. 

Four  case  studies  were  chosen  as  important  examples  of  long-term  research  support  provided 
by  the  AHFMR  to  respected  researchers:  two  clinician  investigators  and  two  basic  scientists. 
The  research  concerned  transplantation  of  pancreatic  islets  of  Langerhans;  ligament  healing; 
electrophysiology  of  the  heart,  and  lipid  metabolism.  Each  was  studied  to  assess  the 
usefulness  and  applicability  of  the  existing  payback  model,  or  possible  variants  of  it. 

It  was  concluded  that  the  research  sequence  model  provided  a  useful  framework  for  reviewing 
what  had  been  the  benefits  from  the  research,  and  that  there  was  particular  merit  in  focussing 
on  the  two  interfaces.  The  first  interface  is  between  the  process  of  defining  research  'needs' 
and  the  undertaking  of  a  specific  piece  of  research.  None  of  the  four  studies  had  been 
commissioned:  it  was  the  researchers  themselves  who  identified  the  original  topics  and  key 
research  sub-topics  and  objectives  at  each  stage  of  their  work.  Typically,  the  researchers' 
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original  interest  in  the  broad  topic  or  theme  arose  through  happenstance.  What  also  came  to 
light  was  that  some  researchers  moved  their  original  curiosity-driven,  or  health  needs-led 
positions.  For  the  clinicians,  moving  towards  basic  science  may  have  been  a  recognition  of  the 
need  to  understand  mechanisms  by  trying  to  determine  the  molecular  basis  of  disease.  For 
basic  scientists  the  focus  on  clinical  application  may  have  reflected  not  only  a  real  interest  in 
improving  health,  but  also  a  recognition  of  the  access  it  provides  to  alternative  sources  of 
funding. 

There  were  important  differences  in  the  subsequent  process  of  defining  and  refining  research 
strategy  and  objectives,  with  differing  degrees  of  involvement  of  the  broader  research 
community.  In  none  of  the  cases  studied  did  there  seem  to  have  been  a  significant  research 
program  specific  interaction  between  the  researcher  and  the  AHFMR,  despite  the  latter's  use  of 
international  scientific  reviewers  for  proposals,  and  the  availability  of  these  reviews  to  the 
researcher.  Although  researchers  welcomed  this  lack  of  interference,  an  opportunity  to 
productively  influence  research  direction  or  design  may  have  been  lost. 

The  second  interface  relates  to  the  way  the  research  is  disseminated,  or  fed  into  the 
international  reservoir  of  knowledge.  The  researchers  interviewed  expressed  very  different 
attitudes  to  this  process.  For  some,  publications  were  the  sole  measure  of  communication  and 
success:  others  emphasised  additional  mechanisms  and  channels.  These  seemed  to  be 
particularly  important  for  those  who  were  personally  concerned  to  maximise  the  impact  of  the 
research  in  terms  of  downstream  applications.  This  has  important  implications  for  the  value  and 
relevance  of  bibliometric  indicators. 

These  case  studies  confirmed  the  importance  of  understanding  better  the  feedback  loops, 
through  which  research  feeds  into  other  research.  If  the  model  were  to  be  used  for  basic 
research,  then  it  would  seem  important  to  begin  to  conceptualize  how  these  feedback  loops 
operate,  and  to  categorize  what  is  being  fed  back.  Focusing  on  the  'dissemination'  interface 
through  which  these  feedbacks  occur  could  perhaps  develop  these  ideas.  This  is  particularly 
important  when  the  feedback  is  into  research  being  undertaken  by  quite  separate  groups, 
possibly  from  a  different  discipline.  In  order  for  research  ideas  to  migrate  from  the  basic  science 
bench  to  application  in  the  clinic,  it  may  need  to  be  disseminated  out  of  the  narrow  specialty  into 
the  thinking  of  a  more  disparate  scientific  audience. 
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Thus,  while  the  case  studies  indicated  that  there  are  significant  difficulties  in  attempting  to 
analyze  the  payback  from  more  basic  research,  they  showed  that  the  'payback'  model  provides 
a  framework  for  posing  some  key  questions  and  for  structuring  ideas  and  indicators  of  payback. 

SECTION  6:  THE  NEXT  STEPS 

From  a  short  period  of  study  of  a  very  complex  question,  it  is  possible  to  draw  a  number  of 
tentative  conclusions  and  proposals  for  further  work. 

immediate  use  by  tiie  AHFIVIR 

•  The  Buxton/Hanney  approach  could  be  applied  to  all,  or  at  least  a  large  sample  of  the 
projects  undertaken  with  the  HTA  Unit's  program  of  work.  The  objectives  of  this  would  be: 

•  to  provide  a  monitoring  device  and  a  process  of  accountability, 

•  to  begin  to  indicate  which  circumstances  or  types  of  projects,  have  the  best  'payback', 
as  an  aid  to  future  prioritization. 

•  For  AHFMR-funded  HSR,  selective  case  studies  could  be  undertaken  to  see  whether 
opportunities  exist  for  the  AHFMR  to  help  researchers  achieve  a  better  payback  on  their 
research. 

•  The  AHFMR  could  review  the  routine  data  collected  on  HSR  projects  to  see  whether  it  could 
easily  be  adjusted  to  provide  better  routine  indications  of  payback. 

•  It  might  consider  whether  its  dissemination  efforts  could  focus  more  on  achieving  the  key 
'secondary  outputs'  from  projects. 

•  For  basic  biomedical  and  clinical  science  research,  it  might  review  whether  the  full  potential 
of  the  interaction  between  researchers  and  scientific  reviewers  (interface  a)  is  being 
realized. 

Furtiier  deveiopment  work  by  thie  AHFiVIR 

•  It  would  be  valuable,  using  citations  analysis,  to  try  to  map  the  pathways  through  which 
specific  research  results  have  been  taken  up  in  other  research,  particularly  attempting  to  link 
through  to  specific  applications. 

•  Additional  case  studies  should  be  undertaken  relating  to  the  use  of  the  model  for  analyzing 
clinical  and  basic  science.  It  would  be  particularly  useful  to  explore  some  projects  that  have 
been  perceived  as  relative  failures,  to  see  whether  this  analytical  approach  could  help 
identify  whether  they  have  generated  little  payback,  and  if  so  why  that  may  have  been. 
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•    The  AHFMR  may  wish  to  set  up  a  seminar  to  encourage  a  wider  local  debate  about,  and 
commitment  to  the  importance  of,  these  issues. 

The  broader  research  issues 

It  remains  unclear  how  best  the  bibliometric  approaches  and  the  more  qualitative 
Buxton/Hanney  approach  can  most  usefully  complement  each  other.  Further  systematic  work  is 
needed. 

There  is  a  key  on-going  disagreement,  between  those  who  assert  that  research  is  unarguably 
beneficial  in  the  long  term  and  should  not  be  assessed  in  terms  of  short-term  benefits,  and 
those  who  argue  that  research  needs  to  be  justified  in  terms  of  its  contribution  to  specific 
shorter-term  objectives.  This  argument  has  been  played  out  in  a  number  of  political  contexts  at 
least  since  the  1960's,  amid  concerns  from  the  research  community  that  fiscal  restraint  and 
government  accountability  may  disadvantage  basic  research  through  failing  to  recognize  its 
long-term  benefits. 

What  is  needed  is  a  vigorous  international  debate  about  the  best  methods  for  ensuring 
accountability  of  research,  for  ensuring  value  for  money  from  research  programs,  and  for 
deciding  whether  or  nor  to  prioritise  research,  be  it  applied  or  basic.  In  the  absence  of 
approaches  that  are  tested,  evidence-based  and  acceptable  to  the  broad  research  community, 
political  and  bureaucratic  pressures  will  see  the  introduction  of  methods  that  have  not  been 
tested,  are  not  evidence-based,  and  may  well  not  command  the  support  of  the  research 
community. 


8 


SECTION  1: 


INTRODUCTION  AND  TERMS  OF  REFERENCE  OF  THE  STUDY 
Background 

Health  services  internationally  are  placing  increasing  emphasis  on  ensuring  that  medicine  in 
particular  and  health  care  delivery  more  generally,  is  evidence-based.  For  those  concerned 
with  promoting  health  related  research,  this  movement  is  very  positive  in  focussing  attention  on 
the  need  for  a  research  foundation,  and  it  enhances  the  potential  contribution  that  relevant 
research  can  make.  However,  this  same  evidence-based  movement  places  an  onus  on  the 
research  community  to  provide  evidence  of  the  value  of  its  own  activities,  and  highlights  the 
reality  that  the  mere  existence  of  evidence  is  not  sufficient  to  ensure  changes  in  policy  and 
practice.  In  aggregate,  it  points  to  growing  interest  in  methods  for  assessing,  retrospectively  or 
prospectively,  the  value  of  research  undertaken  or  proposed. 

Following  an  initial  approach  by  the  Research  and  Development  Directorate  of  the  UK 
Department  of  Health  (DH)  in  1993,  a  small  team  at  the  Health  Economics  Research  Group, 
Brunei  University,  began  working  to  consider  how  best  the  'payback'  or  benefits  of  DH  funded 
research  projects  might  be  conceptualized,  described  and  measured.  This  work  has  now  led  to 
a  series  of  reports  (6,7,9,10),  and  shorter  conference  presentations  and  journal  publications 
(8,1 1).  It  provides  in  particular  a  conceptualization  of  the  benefits  arising  from  health  services 
research  in  terms  of  a  number  of  categories  of  payback  (see  Figurel),  and  a  model  of  payback 
which  places  the  research  itself  within  a  longer  sequence  beginning  with  the  processes  of 
research  needs  assessment  and  going  on  to  the  use  of  research  and  the  benefits  that  arise 
from  its  application  (Figure  2).  These  are  referred  to  in  this  paper  as  the  Buxton/Hanney 
payback  categories  and  the  Buxton/Hanney  payback  model.  Using  these  as  frameworks  for 
analysis,  the  Brunei  team  have  undertaken  and  published  some  18  ex  post  case  studies, 
analyzing  research  projects  or  programs  undertaken  in  the  last  few  years  (6,10).  Currently  the 
team  is  undertaking  work  for  North  Thames  NHSE  R&D  Directorate  reviewing  research  and 
related  educational  activities  that  it  has  commissioned  in  the  last  few  years  with  a  view  to 
establishing  a  routine  system  for  monitoring  'payback'  achieved. 

To  date  this  work  by  the  HERG  Team  has  all  been  undertaken  within  the  context  of  the  UK  and 
the  published  case  studies  have  all  been  applied  health  services  research,  or  methodological 
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development  for  use  in  applied  health  services  research.  However,  there  has  been  a 
considerable  amount  of  interest  in  the  wider  applicability  of  this  work  both  to  HSR  undertaken  in 
other  countries  or  settings,  and  to  basic  science  or  clinical  research  that  is  some  way  from  being 
of  immediately  applicable  to  health  policy  and  or  practice.  Following  interest  expressed  by  the 
AHFMR  in  this  work,  it  was  agreed  that  Martin  Buxton  should  spend  a  short  period  working  at 
the  AHFMR,  to  test  whether  the  model  might  be  useful  to  the  AHFMR  in  reviewing  the  payback 
achieved,  or  achievable,  from  the  research  it  funds. 

The  aim  of  this  particular  project  was  not,  at  this  stage,  to  assess  the  overall  payback  from 
AHFMR-funded  research,  nor  to  'audit'  individual  projects.  Either  of  these  would  require  that 
thorough  and  systematic  case  studies  be  undertaken  on  all,  or  a  formal  sample  of,  funded 
research.  Neither  would  have  been  feasible  within  the  time  available,  nor  appropriate  without 
some  initial  testing  of  the  model  in  this  context.  Instead  the  work  focused  on  this  initial  testing. 

Terms  of  Reference 

There  were  two  broad  objectives  to  this  phase  of  model  testing: 

•  The  first  was  to  confirm  that  an  approach  developed  in  the  UK  for  the  Department  of 
Health  to  help  them  look  at  the  benefits  gained  from  health  services  research 
projects  could  usefully  and  meaningfully  be  applied  to  similar  applied  research 
funded  through  the  AHFMR. 

•  The  second  objective,  more  speculative  and  challenging,  was  to  see  whether  the 
approach,  or  an  adaptation  of  it,  could  be  applied  to  funding  focused  on  less  applied 
areas  of  health  research  such  as  basic  and  clinical  science. 

The  additional  factor,  and  in  essence  a  third  test  to  the  model,  was  that  in  both  instances  the 
approach  had  to  work  in  the  context  of  where  most  of  the  AHFMR's  funding  is  focused.  That  is 
primarily  to  support  individuals,  or  teams,  as  part  of  a  process  of  capacity  building,  rather  than 
simply  short-term  funding  of  specific  individual  projects. 

In  the  light  of  this  testing  the  agreed  remit  was  to  make  some  initial  recommendations  as  to  how 
most  usefully  the  AHFMR's  interest  in  this  broad  issue  might  be  pursued  and  how  the 
techniques  and  methods  might  best  be  applied  and  developed. 
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a)  Knowledge 

b)  Benefits  to  future  research  and  research  use: 
i)      the  better  targeting  of  future  research; 

//)      the  development  of  research  skills,  personnel  and  overall  research 
capacity; 

Hi)     a  critical  capability  to  utilize  appropriate  existing  research  including 

that  from  overseas; 
iv)    staff  development/educational  benefits. 

c)  Political  and  administrative  benefits: 

i)  improved  information  bases  on  which  to  take  political  and  executive 
decisions; 

ii)  other  political  benefits  from  undertaking  research. 

d)  Health  sector  benefits: 

i)  cost  reduction  in  the  delivery  of  existing  services; 

ii)  qualitative  improvements  in  the  process  of  service  delivery; 
Hi)     increased  effectiveness  of  services  eg.  increased  health; 

iv)    equity  eg.  improved  allocation  of  resources  at  an  area  level,  better 

targeting  and  accessibility; 
iv)     revenues  gained  from  Intellectual  Property  Rights. 

e)  Broader  economic  benefits: 

i)  wider  economic  benefits  from  commercial  exploitation  of  innovations 
arising  from  R&D; 

ii)  economic  benefits  from  a  healthy  workforce  and  reduction  in  working 
days  lost. 


Figure  1:  The  Buxton/Hanney  'categories  of  payback' 
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SECTION  2: 


METHODS 

Most  of  the  work  was  undertaken  in  a  two-month  period  during  mid-June  to  mid-August,  1998, 
and  completed  in  October  1998.  Professor  Martin  Buxton  (of  the  Health  Economics  Research 
Group,  Brunei  University)  who  had  led  the  UK  studies  on  which  this  project  was  based,  and 
Wendy  Schneider  from  the  AHFMR  undertook  the  work.  Within  the  limited  time  available,  it  was 
agreed  to  use  a  selective  case  study  approach  in  which  the  case  studies  would  be  developed 
only  as  far  as  was  necessary  to  begin  to  test  the  applicability  of  the  model.  It  was  not  feasible, 
and  was  never  the  intention,  to  make  the  case  studies  exhaustive  accounts  of  the  research  in 
question.  It  was  the  approach  that  was  being  tested,  neither  the  research  nor  the  researchers, 
chosen  as  test  cases. 

The  first  step  was  to  attempt  to  get  a  reasonable  understanding  of  the  context  of  the  work  of  the 
AHFMR  and  of  the  health  system  in  Alberta,  with  a  concern  to  establish  whether  there  seemed 
to  be  any  particular  charactenstics  which  might  cause  problems  in  applying  the  existing 
Buxton/Hanney  approach.  This  was  achieved  through  review  of  earlier  studies  undertaken  in 
Alberta  pertaining  to  this  broad  question,  and  background  information  on  the  AHFMR,  the 
Alberta  Health  Care  system,  through  informal  discussions  with  the  AHFMR's  staff  and  with  a 
number  of  researchers  involved  in  health  research.  Additional  insights  came  through  discussion 
with  others  involved  in  the  health  service  and  analysis  of  the  AHFMR's  funding  processes 
relating  to  HSR.  In  addition  Wendy  Schneider  brought  first-hand  knowledge  and  experience  of 
the  local  system. 

To  test  whether  the  Buxton/Hanney  approach  could  readily  be  applied,  two  case  studies  were 
chosen  initially  as  good  examples  of  applied  HSR  funded  by  the  AHFMR.  As  such,  they  had 
already  been  fairly  well  documented.  Indeed,  the  work  by  one  researcher  had  been  written  up 
as  a  case  study  in  the  report  by  Kerry  Toll  (35).  It  was  largely  possible  to  undertake  both  case 
studies  to  the  extent  necessary  to  test  the  model  using  materials  on  file:  research  applications, 
reviewers  comments,  annual  reports  and  copies  of  main  publications  and  secondary  accounts 
of  the  research  to  the  extent  necessary  to  test  the  model.  Interviews  with  the  researchers  were 
done  to  verify  the  conclusions  drawn  from  the  file  materials  and  to  identify  other  categories  of 
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payback.  In  addition  to  tlnese  case  studies  of  funded  HSR,  two  case  studies  (3a  and  3b)  were 
undertaken  on  Health  Technology  Assessment  (HTA)  projects,  part  of  the  work  of  the  AHFMR's 
HTA  program.  These,  too,  were  undertaken  principally  from  on-file  documentation,  and  with  the 
first  hand  knowledge  of,  and  access  to,  the  staff  researchers  concerned. 

The  second  and  more  innovative  part  of  the  work  involved  establishing  whether  the 
Buxton/Hanney  approach,  or  a  variant  of  it,  could  usefully  be  applied  to  biomedical  and  clinical 
research.  For  this  it  was  agreed,  in  this  first  phase  to  use  a  number  of  case  studies  to  test  the 
model  and  focus  thinking  about  the  problems  posed  by  more  basic  science.  On  the  advice  of 
the  AHFMR's  staff,  four  distinguished  researchers,  from  the  Universities  of  Alberta  and  Calgary, 
were  chosen  each  of  whom  clearly  was  acknowledged  as  a  leader  in  their  respective  fields,  and 
whose  research  is  of  the  highest  standard.  The  first  two  were  Clinical  Investigators:  Dr.  "A",  and 
Dr.  "B".  The  second  two  were  basic  biomedical  scientists:  Dr.  "C"  and  Dr.  "D".  All  four  had 
received  extensive  AHFMR  funding  over  a  number  of  years.  In  each  case  the  mode  of  working 
was,  again,  initially  to  undertake  a  desk  review  of  materials  on  file  and  then  to  interview  the 
researchers  themselves  and  to  explore  the  way  that  they  perceived  and  conceptualized  the 
'payback'  on  their  own  research  and  to  consider  how  this  payback  might  be  assessed.  For 
these  four  interviews,  a  common  structure  of  broad  questions  was  used,  although  general 
questions  were  individualized  somewhat  as  a  result  of  the  initial  desk  review  of  their  research. 
The  general  questions  are  set  out  in  Figure  3.  Due  to  time  constraints,  it  was  not  possible  to 
follow-up  issues  raised  in  these  interviews  with  others,  such  as  those  who  may  have  used  the 
research  findings,  and  hence,  once  again,  the  case  studies  should  not  be  seen  as  definitive 
analyzes  of  the  payback  from  these  streams  of  research.  They  simply  provide  an  initial 
indication  of  how  the  model  might  be  used  in  such  contexts. 

In  parallel  with  the  case  studies,  a  continuing  review  of  relevant  literature  on  the  evaluation  of 
the  benefits  of  research,  particularly  basic  science  research  was  undertaken.  While  at  this 
stage  this  does  not  represent  a  systematic  review  of  that  diffuse,  and  difficult  to  access,  existing 
knowledge  base,  our  work  does  draw  on  some  of  this  research,  by  way  of  ideas  or  contrasting 
approaches. 
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Figure  3:  Interview  questions  for  researchers 


•     What  was  the  original  impetus  for  this  program  of  research? 


•     What  drives  your  research.  Scientific  curiosity?  The  need  to  fill  certain  gaps  in  knowledge? 
The  inadequacy  of  current  treatments? 


•     What  are  your  main  research  objectives  and  how  will  they  be  important  scientifically  and 
clinically?  What  do  you  see  as  the  main  expected  benefits  of  your  planned  research? 


•     How  internationally  competitive  is  this  area  and  how  collaborative? 


•     What  kind  of  interaction  is  involved  in  determining  your  research  plans?  With  funders?  With 
University  colleagues?  With  peers  internationally  in  specific  research  communities? 


•     How  do  you  see  the  results  of  your  immediate  research  being  used  and  by  whom? 


•     What  are  the  really  important  ways  in  which  results  and  ideas  get  shared  between  the  main 
players  in  the  research  community? 


•     Have  other  centres  picked  up  and  further  developed  ideas  stemming  from  your  research? 


•     Have  others  adopted  specific  techniques  or  tools  you  have  developed? 


•     Have  the  plans  for  your  program  been  affected  by  results  from  elsewhere? 


•     How  will  these  results  feed  through  into  the  development  of  improved  clinical 
therapy/treatment? 


•     What  are  the  key  steps  in  this  process,  or  links  in  this  chain? 


•     Realistically  what  sorts  of  time  scales  are  likely  to  be  involved? 


•     How  many  years  are  you  from  having  an  established  proven  clinical  therapy  that  might  be 
adopted  by  clinical  centres  elsewhere? 


•     What  do  you  see  as  the  best  indicator  of  the  importance  or  value  of  research  such  as  this? 


•     Is  publication  in  the  major  peer-review  journals  the  key  channel  or  an  indicator  of  quality  of 
the  work? 


•     What  for  you  personally  are  the  key  indicators  of  success? 


•     What  features  of  your  situation  at  this  University  are  particular  advantages/disadvantages  to 
pushing  this  research  forward  successfully? 


MJBand  WLS  14/7/98 
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SECTION  3: 


IS  THE  WORK  ON  METHODS  FOR  ASSESSMENT  OF  RESEARCH  BENEFITS 
INTERNATIONALLY  GENERALIZABLE  OR  IS  AHFMR  OR  ITS  CONTEXT 
DIFFERENT? 

Introduction 

There  appears  to  be  a  cycle  of  interest  in  establishing  the  benefits  of  research.  At  one  point  in 
time  those  who  assert  that  research  is  unarguably  beneficial  in  the  long-term,  and  an 
undeniable  public  good,  may  be  in  the  ascendant.  At  another  point  of  time  those  who  argue  that 
spending  on  research  needs  to  be  justified  in  terms  of  its  contribution  to  specific  shorter-term 
benefits  have  a  stronger  pull  on  the  public  purse-strings.  In  reviewing  some  of  the  international 
literature,  it  is  interesting  to  see  how  the  concerns  of  the  US  academics  in  the  late  1960's  about 
the  stance  of  President  Lyndon  Johnson.  His  statement  that  basic  science  needed  to  be  more 
targeted  and  the  time  from  laboratory  discovery  to  useful  application  reduced  (14)  are  mirrored 
in  the  recent  concern  expressed  by  the  Federation  of  American  Societies  for  Experimental 
Biology  (17).  Their  concern  was  that  the  1993  Government  Performance  and  Results  Act  might 
disadvantage  basic  research  through  failing  to  recognize  its  long-term  benefits. 

In  drawing  on  this  international  literature,  it  is  tempting  to  presume  that  method  for  the 
assessment  of  benefits  or  value  derived  from  research  might  be  internationally  generalizable. 
While  the  earlier  Buxton/Hanney  work  was  developed  specifically  in  the  UK  context,  it  certainly 
drew  on  lessons  from  studies  undertaken  elsewhere.  For  example,  its  consideration  of  the 
policy  impact  was  informed  by  a  stream  of  work  undertaken  elsewhere  in  Canada  reviewing  the 
policy  impact  of  the  Quebec  Council  on  Health  Care  Technology  Assessment  (22-25).  This 
work  appears  to  reflect  very  similar  policy  concerns  to  those  in  the  UK.  It  comes  to  rather 
similar  conclusions  about  the  difficulty  of  precisely  estimating  impact  but  the  systematic 
descriptive  analysis  of  impact  is  both  feasible  and  worthwhile. 

But  such  presumptions  need  to  be  checked  for  in  Health  Services  Research  there  is  a  frequent 
concern  that  research  findings  may  not  be  geographically  generalizable.  For  example,  a  new 
technology  may  be  cost-effective  in  one  country  or  context  but  not  in  another,  or  a  particular 
form  of  incentive  may  be  powerful  in  one  health  care  system  but  ineffective  in  another. 
Generally,  however,  methods  of  analysis  are  transferable,  although  even  these  may  be 
culturally  influenced.  For  example,  willingness-to-pay  as  a  way  of  obtaining  individuals' 
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valuations  of  health  states  has  tended  to  be  much  more  readily  accepted  and  practically  valid  in 
the  USA.  There,  patients  are  more  used  to  making  direct  payments  for  health  care,  than  in  the 
UK,  where  patients  have  different  cultural  expectations  and  experience. 

Thus,  it  was  a  reasonable  precaution  to  consider  as  a  first  step  whether  there  was  evidently  any 
reason  why  the  nature  and  objectives  of  the  AHFMR,  or  the  health  care  and  health  research 
systems  in  Alberta,  should  make  the  Buxton/Hanney  approach  invalid  or  inappropriate  in  this 
new  context. 

This  section  begins  by  very  briefly  reviewing  a  number  of  characteristics  of  the  local  context  but 
focuses  principally  on  analysis  of  the  explicit  and  implicit  objectives  of  health  research  in 
Alberta.  It  highlights  a  number  of  ways  in  which  the  work  of  the  AHFMR  does  differ  from  that  in 
the  UK  for  which  the  model  was  originally  created. 

Is  the  Alberta  Health  Care  System  So  Different? 

In  this  brief  period  of  study,  no  attempt  was  made  systematically  to  compare  the  health  care 
systems  of  the  UK  and  Alberta.  However,  a  cursory  examination  suggests  that  for  the  most  part 
the  important  characteristics,  from  the  perspective  of  how  one  might  consider  the  payback  on 
health  services  research,  are  common  to  both.  Other  writers  have  looked  at  the  use  of  research 
in  the  development  of  health  policy  in  Canada  and  the  UK,  and  while  the  pictures  they  present 
differ  in  detail  many  common  themes  emerge  (26,28). 

This  is  not  surprising,  for  the  two  systems  share  much  in  common:  not  least  the  importance  of 
public  provision  and  financing  of  much  health  care.  They  share  the  same  current  concerns 
about  increasing  the  efficiency  of  health  care  provision,  of  ensuring  equity  of  access  and  of 
striving  towards  a  stronger  evidential  basis  for  health  policy  and  practice.  Similar  barriers  exist: 
democratically  elected  politicians  in  both  systems  appear  to  work  to  equally  short  time-scales. 
Health  professionals  resent  external  intrusion  and  monitoring,  and  prefer  self-regulation.  Past 
practice,  convention  and  recent  experience  are  as  strong  moderators  of  behaviour  as  the  most 
powerful  of  meta-analyses.  The  public  in  both  countries  are  fascinated  by  what  science  can 
offer  and  expect  new  developments,  often  heralded  by  the  media  well  in  advance  of  their  being 
ready  for  public  use,  to  be  immediately  available  as  an  addition  to  the  range  of  health  care 
already  on  offer  to  them.  More  specifically  the  debate  about  the  need  to  justify  spending  on 
research  appears  to  be  very  similar  in  Alberta  as  in  the  UK  and  elsewhere. 
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It  is  important,  however,  to  recognize  the  difference  in  population  sizes,  with  Alberta  having  only 
1/20*''  of  the  population  of  the  UK.  Thus,  Alberta  has  a  population  considerably  smaller  than  any 
of  the  UK  health  regions,  albeit  in  a  land  mass  nearly  three  times  the  size  of  the  UK.  This  has 
some  important  implications  for  the  research  debate.  In  terms  of  excellence  in  research  itself,  it 
would  require  a  much  larger  investment  per  capita  to  achieve  the  same  absolute  levels  of 
international  excellence  as  that  of  a  much  larger  country.  Thus,  there  is  likely  to  be  an  even 
stronger  debate  than  in  the  UK  as  to  whether  it  is  justified  to  undertake  particular  research 
locally  or  whether  it  is  better  to  free-ride  on  international  research  findings.  And,  more  like  the 
debate  at  a  Regional  level  in  the  UK,  there  may  be  arguments  about  the  value  of  the 
involvement  of  health  professionals  locally  in  non-internationally  competitive  research,  as  a  way 
of  enhancing  their  receptivity  to  research  findings  generated  elsewhere  (7). 

But,  while  recognizing  many  important  differences  of  details,  an  informal  review  of  the  system 
indicated  that  there  were  no  major  structural  or  cultural  differences  that  might  invalidate  the 
underpinnings  of  the  Buxton/Hanney  approach.  There  seems  every  reason  to  expect  the 
framework  to  be  relevant. 

That  is  not  to  say  that  there  may  not  be  differences  in  what  enhances,  or  detracts  from,  the 
payback  on  particular  research.  For  example,  the  smaller  scale  may  make  it  easier  for 
researchers  to  make  the  necessary,  on  going  linkages  to  policy-makers  (33).  On  the  other 
hand,  there  may  not  be  local  researchers  who  can  pick-up  more  basic  research  and  take  it  a 
step  further  towards  application.  Instead,  it  may  be  necessary  to  ensure  the  international 
dissemination  of  research  ideas  and  then  to  be  able  to  bring  them  back  into  the  province  when 
others  have  developed  them  further.  Thus,  the  empirical  findings  using  the  model  might  indeed 
differ  somewhat. 

The  local  objectives  for  research 

As  with  any  system  of  evaluation  or  performance  measurement,  it  is  essential  that  the 
evaluation  criteria  or  performance  measures  reflect  as  closely  as  possible  the  objectives  of  the 
activity  in  question.  Thus,  one  factor  that  might  lead  to  different  conclusions  about  research 
might  be  if  there  were  different  objectives,  or  if  within  the  same  overall  set  of  objectives, 
priorities  and  relative  emphases  differed  significantly.  Sometimes  those  objectives  are  implicit 
rather  than  explicit  and  frequently  they  may  be  complex  and  multifaceted.  While  explicit  mission 
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statements  may  not  adequately  articulate  the  true  complexity  of  objectives,  they  are  the  obvious 
and  appropriate  starting  point. 

The  Alberta  Heritage  Foundation  for  Medical  Research  (AHFMR)  has  the  following  mission 
statement: 

We  support  a  community  of  researchers  wtio  generate  knowledge  that  improves  the 
health  and  quality  of  life  of  Albertans  and  people  throughout  the  world.  Our  long-term 
commitment  is  to  fund  basic,  patient  and  health  research  based  on  international 
standards  of  excellence  and  carried  out  by  new  and  established  investigators  and 
researchers  in  training. 
Created  by  Legislation  in  1979,  the  purpose  of  the  AHFMR  was  to  establish  and  support  a 
balanced,  long-term  program  of  medical  research  based  in  Alberta.  The  AHFMR  stands  at 
arms-length  from  the  Provincial  Government  and  is  supported  by  an  endowment  fund  that 
generates  interest  revenue  for  operating  expenses. 

In  addition,  under  the  Health  Research  Collaboration,  Alberta  Health  contracted  its  health 
research  roles  and  responsibilities  to  the  AHFMR  in  three  main  areas.  It  gave  a  granting 
agency  role  for  the  Health  Research  Fund,  a  research  institute  role  for  Health  Technology 
Assessment,  and  a  research  policy  role  to  develop  a  Health  Research  Agenda  and  health 
research  priorities. 

Alberta  Health's  mission  is: 

to  enhance  the  public  and  health  providers  to  protect,  maintain,  restore  and  enhance  the 
health  of  Albertans. 

The  goals  of  their  health  research  mandate  have  been  expressed  as  follows: 

•  competent  researchers  work  in  Alberta,  undertaking  research  in  support  of  health 
system  goals; 

•  effectiveness  and  efficiency  of  the  health  system  is  evaluated; 

•  broad  awareness  of  research  results. 

In  an  Alberta  Health  communication  to  the  Alberta  Science  and  Research  Authority  (ASRA),  the 
expected  outcomes  are  phrased  in  terms  of  impact  as  follows: 
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•  the  application  of  research  results  improves  the  health  status  of  individuals  and  the 
population  as  a  whole; 

•  health  system  sen/ices  become  more  efficient  and  effective  as  a  result  of  the 
evaluation  of  inten/entions,  programs  and  services; 

•  evidence  based  decision  making  contributes  to  the  effective  management  of  the 
health  system; 

•  expertise  in  health  services  is  enhanced  and  applied  in  support  of  health  system 
goals; 

•  research  results  are  broadly  disseminated,  increasing  awareness  of  research  results. 

In  turn  the  mission  of  ASRA  is: 

to  enhance  the  contribution  of  science  and  research  to  the  sustainable  prosperity  and 
quality  of  life  of  all  Albertans. 

The  Implications  of  the  Alberta  Context 

These  various  mission  statements  and  descriptions  of  objectives  and  goals  are  in  no  way 
exceptional  and  could  apply  to  similar  institutions  in  many  countries.  Each  one  of  the  objectives 
stated  above  points  to  an  aspect  of  payback  already  included  in  the  Buxton/Hanney  categories. 
Indeed,  the  broader  context  of  a  major  commitment  to  health  research,  but  with  an  increasing 
need  to  justify  that  commitment,  reflects  a  common  situation  in  many  parts  of  the  world,  and  one 
certainly  similar  to  that  in  the  UK  where  the  model  was  developed. 

However,  there  may  be  some  important  differences  in  emphasis  in  Alberta  that  need  to  be 
reflected  in  work  on  payback.  The  first  is  the  recognition  that  the  development  of  substantial 
research  interests  in  health  sciences  is  relatively  recent  in  Alberta.  Much  of  what  the  AHFMR 
has  been  doing  in  recent  years  has  been  developing  a  capacity  and  building  an  infrastructure 
for  health  research  based  principally  on  the  two  main  and  competing  universities  (University  of 
Calgary  and  the  University  of  Alberta,  in  Edmonton).  This  means  that  there  has  been  an 
emphasis  on  payback  in  terms  of  Category  bii):  'development  of  research  skills,  personnel  and 
overall  research  capacity'.  Thus,  much  of  the  early  research  has  aimed  to  build  research  teams 
to  encourage  and  develop  younger  research  staff  and  to  equip  them  to  begin  to  be  competitive 
at  a  national  and  international  level.  The  implication  is  that  it  has  been  recognized  that  the 
health  paybacks  would  occur  in  the  longer  term  from  subsequent  phases  in  this  strategic 
development.  It  has  also  meant  that  research  funding  has  been  seen  as  having  a  broader 
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educational  role,  in  helping  educate  future  researchers.  These  together  have  meant  that  the 
AHFMR  has  an  indirect  but  very  significant  objective  to  foster  the  success  of  the  two 
Universities  in  relation  to  health  research. 

Another  factor  informing  research  investment  is  the  AHFMR's  own  periodic  review  processes. 
Under  the  provisions  of  the  AHFMR  Act,  an  International  Board  of  Review  (IBR)  is  required  to 
appraise  the  operation  of  the  AHFMR  at  intervals  of  six  years,  and  report  its  recommendations 
to  the  Trustees  of  the  AHFMR.  The  AHFMR  has  undergone  three  of  these  reviews,  in  1987, 
1992,  and  1998.  The  review  process  assembles  information  bearing  on  some  of  the  aspects  of 
payback  in  the  Buxton-Hanney  model  (categories  a,  b  (i  &  ii),  and  e,  (i)).  The  information 
informs  the  forward  planning  processes  of  the  AHFMR. 

The  Universities  in  turn  may  be  rather  more  sensitive  than  many  other  Universities  to  the  need 
to  show  the  broader  value  of  the  investment  in  them.  For  example  the  Vice  President  for 
Research  at  the  University  of  Calgary  (Dr  Cooper  Langford)  commissioned  internal  studies  of 
the  economic  benefits  to  the  Province  of  Alberta,  and  of  the  influence  on  policy,  provided  by  the 
Faculty  of  the  University  (12,37).  Both  these  reports  provide  evidence  of  the  impact  on  policy- 
making by  members  of  the  Faculty  of  Medicine,  which  was  judged  as  being  the  most  productive 
in  the  University  with  respect  to  activities  leading  to  economic  development. 

An  important  part  of  the  strategic  context  for  the  major  investment  by  the  AHFMR  in  health 
research  is  the  Provincial  concern  that,  as  Alberta  moves  into  the  21^*  century,  there  cannot  be 
continued  reliance  on  commodity-based  prosperity.  A  high  level  of  innovation  is  required  based 
upon  investments  in  science  and  R&D,  and  there  is  concern  that  in  terms  of  innovative  capacity 
Alberta  appears  to  be  falling  behind  the  rest  of  Canada  and  the  world  (14).  More  recently  a  task 
force  was  set  up  in  the  province  which  concluded: 

Alberta  has  a  solid  base  for  health  research  and  for  a  health  research  industry. 
Development  of  a  robust  health  research  industry  represents  a  strategic  economic 
opportunity  for  this  province.  This  industry  has  significant  potential  to  capitalize  on  a 
strong  foundation  to  advance  the  creation  of  knowledge,  to  improve  quality  of  life  and  to 
increase  prosperity  for  Albertans  by  creating  high  quality  jobs  and  a  more  productive  and 
skilled  workforce  (1). 
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This  suggests  that  the  broad  economic  benefits  from  research  [Buxton/Hanney  Category  ei)] 
may  be  of  equal  or  greater  importance  for  this  strategic  investment  than  the  benefits  to  the 
health  sector  itself.  Thus,  within  the  standard  tensions  between  what  may  be  best  for  the  health 
care  system  (for  example,  the  cautious  acceptance  of  new  technology)  and  what  may  be  best 
for  the  economy  (for  example,  a  strong  home  market  base  to  facilitate  the  export  of  an 
indigenously  developed  technology)  the  balance  in  Alberta  might  be  perceived  as  lying  towards 
concerns  about  the  economy. 

Biopharmaceuticals,  agri-food,  information  technology  and  telecommunications  are  seen  as  key 
R&D  intensive  areas  for  development  in  which  Alberta  wishes  to  recruit  companies  of  sufficient 
size  to  create  critical  mass  in  R&D  (2).  The  Technology  Commercialization  Program  within  the 
AHFMR  exemplifies  the  concern  to  encourage,  if  commercially  feasible,  the  commercial 
development  and  marketing  of  technologies  that  arise  from  the  basic  and  applied  research  that 
the  AHFMR  and  others  have  supported  in  the  Province  (3). 

Finally,  there  seems  to  be  a  broad  acceptance  of  the  considerable  time-scales  such  investment 
entails.  The  Report  "Sustaining  the  Alberta  Advantage"  recognizes  that  the  benefits  from 
investment  in  R&D  occur  over  a  long  time  period  (see  Figure  4).  But  already,  the  biomedical 
disciplines  (cell  and  developmental  biology;  molecular  biology  and  genetics;  immunology,  and 
biochemistry  and  biophysics)  are  seen  as  shining  exceptions  to  the  declining  research  status  of 
Alberta.  In  these  fields,  Alberta  researchers  have  publication  impacts  2-3  times  the  world 
average  (for  the  same  fields)  and  this  success  is  seen  as  demonstrating  the  effectiveness  of  the 
provincial  strategy  of  investing  in  the  AHFMR  (5). 
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The  current  prosperity  that  Albertans  enjoy  can  be  traced  to  waves  of  investment  in 
science  and  research\ 


Figure  4:  Illustration  depicting  R&D  investment  timelines 

(Figure  from  report:  Sustaining  the  Alberta  Advantage  (5),  used  with  permission) 


Abbreviations  in  the  figure:  AOSTRA  -  Alberta  Oil  Sands  Technology  Research  Authority,  AHFMR  - 
Alberta  Heritage  Foundation  for  Medical  Research.  AARI  -  Alberta  Agricultural  Research  Institute.  TRLabs  - 
Telecommunications  Research  Labs,  AMC  -  Alberta  Microelectronics  Centre.  C-FER  -  Centre  for 
Engineering  Research.  AEC  -  Alberta  Environmental  Centre  at  Vegreville 
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The  Range  of  AHFMR  Funding 

What  may  also  be  unusual  is  the  comprehensive  range  of  activities  that  the  AHFMR  supports. 
By  scientific  field,  the  areas  supported  range  from  fundamental  biology  through  clinical 
investigation  to  studies  on  population  health  in  the  community.  The  description  that  follows  is 
the  breakdown  of  expenditures  on  a  total  base  of  $36.6  million  in  fiscal  97/98.  The  single 
largest  component  is  the  Personnel  Support  programs,  which  contribute  to  the  salary  costs  of 
faculty-level  investigators  in  universities  and  affiliated  institutions  (56%).  The  second  largest 
component  is  the  Training  programs  at  17%.  These  include  Summer  Studentship  awards, 
Graduate  Studentships,  and  Postdoctoral  Fellowships.  Both  the  Training  and  Personnel  awards 
are  available  to  qualified  researchers  across  the  entire  spectrum  of  research  investigation  from 
basic  biomedical  research  to  health  services  and  population  health  research.  In  addition,  the 
AHFMR  has  a  variety  of  other  programs  (totaling  12%  of  expenditures)  which  includes  funding 
for  major  equipment  and  facilities,  visiting  lecturer  and  travel  awards,  visiting  professorships, 
infrastructure  grants,  conference  and  workshop  support,  and  the  support  for  special  projects. 

A  Technology  Commercialization  program  accounts  for  some  3%  to  4%  of  expenditures  and 
funds  Alberta  innovators  in  the  transferring  of  new  ideas  in  scientific  findings  into  commercial 
health  care  products  and  processes. 

The  AHFMR  provides  direct  support  for  training  programs  for  the  Regional  Health  Authorities  to 
promote  the  uptake  of  evidence,  and  the  development  of  research  projects  at  the  community 
level  (approximately  2%  of  expenditures).  The  program  involves  both  the  funding  of  training, 
and  the  institutional  support  services  for  consultation  and  methodology  support. 

In  addition,  through  collaboration  with  Alberta  Health,  the  AHFMR  administers  a  Health 
Research  Fund  for  individual  project  support,  a  Health  Technology  Assessment  Unit,  and 
provides  support  for  dissemination  and  health  research  agenda  setting  activities  (6%  of 
expenditures). 
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SECTION  4: 


APPLYING  THE  BUXTON/HANNEY MODEL  TO  AHFMR-FUNDED  HSR 
RESULTS  FROM  THE  CASE  STUDIES 

Introduction 

Given  that  the  Buxton/Hanney  approach  to  assessing  the  payback  from  research  was 
developed  in  the  context  of  HSR  projects  the  first  test  was  simply  to  see  whether  it  could  readily, 
and  usefully,  be  applied  to  similar  HSR  funded  by  AHFMR  in  Alberta.  The  expectation  was  that 
this  would  be  feasible,  particularly  as  the  earlier  review  of  the  context  of  AHFMR  funding 
(Section  3)  had  not  indicated  any  major  local  characteristic  that  might  invalidate  the  approach. 

In  order  to  demonstrate  the  application  of  the  model  two  pieces  of  research  were  initially 
analyzed  as  case  studies  of  externally  funded  HSR: 

•  Case-Study  1 :  Evaluation  of  a  novel  diagnostic  and  treatment  method  for  sleep 
apnea.  Principal  Investigator  -  Dr.  HSR-1 

•  Case-Study  2:  The  value  of  lumbar  spine  radiographs  in  the  management  of  low 
back  pain.  Principal  Investigator  -  Dr.  HSR-2 

In  addition,  two  shorter  examples  of  the  work  of  the  AHFMR's  own  internal  HTA  Unit  was 
analyzed  using  the  model: 

•  Case-Study  3:  HTA  Projects: 

a)  Stereotactic  radiosurgery 

b)  Telephone  nurse  triage  services.  ' 

Each  of  these  is  summarized  briefly  below  using  the  framework  of  the  model:  the  full  accounts 
of  the  case  studies  are  available  upon  request.  A  final  section  then  draws  together  a  number  of 
conclusions  from  this  test  of  the  model  with  these  case  studies. 
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Case-study  1:  Diagnostic  methods  for  obstructive  sleep  apnea  (OSA) 

This  particular  research  work  was  conducted  under  the  leadership  of  Dr.  HSR-1  who  had  been 
supported  since  1984  with  an  AHFMR  'Scientist'  award.  During  this  period  he  developed  a 
world-class  sleep  laboratory.  Thus,  while  it  is  viewed  here  as  a  separately  funded  phase  of 
research,  it  constitutes  part  of  a  longer  and  ongoing  program  of  research. 

Research  Needs  Assessment  The  impetus  for  Dr.  HSR-1  to  begin  this  research  project  came 
out  of  a  feeling  of  an  inability  to  treat  patients  with  OSA  in  a  timely  manner.  Approximately 
80,000  Albertans  and  six  to  nine  million  people  in  North  America  suffer  from  OSA.  At  the  time 
this  study  was  undertaken  there  were  only  2  facilities  in  Alberta  (Edmonton  and  Calgary)  that 
were  able  to  diagnose  sleep  apnea.  The  standard  approach,  the  polysomnogram,  to  diagnose 
and  to  establish  a  treatment  for  these  individuals  typically  requires  one  to  two  nights  of 
monitoring  in  a  sleep  laboratory  at  a  cost  of  approximately  $1,000  to  $2,000  in  a  special  hospital 
based  facility.  Dr.  HSR-1  thought  that  this  technology  was  extremely  expensive  and  inadequate 
in  terms  of  its  ready  availability.  A  simpler  device  for  outpatient  and  home  diagnosis  called 
SNORESAT®  was  developed. 

The  researcher  himself  identified  the  need  for  the  research,  but  patients  dissatisfied  with 
services  available  supported  this  need  locally.  The  Alberta  Solicitor  General's  Office  restricts 
Albertans  suspected  of  having  sleep  apnea  from  driving  until  they  have  been  diagnosed  and 
treated.  The  costs  of  accidents  (in  terms  of  mortality,  injuries  and  financial  costs)  due  to  sleep 
apnea  are  thought  to  be  substantial. 

Interface  (a)  As  is  normal  for  AHFMR-funded  research,  expert  scientific  review  informs  the 
decision  to  fund  the  research.  Dr.  HSR-1  collaborated  with  the  departments  of  Cardiology, 
Engineering  and  Industrial  Design  which  helped  to  shape  the  research  design.  This  research 
was  part  of  an  ongoing  stream  of  work  and  it  is  obvious  that  Dr.  HSR-1  knew  well,  and  was 
known  well  by,  the  key  local  stakeholders. 

Inputs  The  major  specific  'input'  was  a  4-year  grant  totaling  $231,544  awarded  in  1993  from 
the  Health  Services  Research  and  Innovation  Fund.  The  work  also  drew  on  the  existing 
facilities  of  the  sleep  laboratory. 
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Processes  The  main  element  of  the  study  was  a  randomized  clinical  trial  involving  300 
participants.  In  the  trial,  the  SNORESAT®  was  compared  with  the  gold  standard:  a 
polysomnogram.  The  purpose  of  the  research  was  to  change  medical  practice,  specifically  by 
developing  a  device  that  efficiently  diagnoses  sleep  apnea  without  the  need  of  a  hospital  stay. 

Primary  Outputs  Three  papers  documenting  results  for  different  aspects  of  this  study  were 
submitted  to  international  journals.  Examples  of  journals  that  have  published  Dr.  HSR-1's  work 
are  the  American  Review  of  Respiratory  Disease.  148(4  Ft  1):1023-9,  1993  Oct.,  Journal  of 
Applied  Physiology  83{3):851-9,  1997  Sept.;  and  Sleep  19(10  Suppl):S243-7,  1996  Dec.  The 
results  have  also  fed  directly  into  further  research  by  the  same  team. 

Interface  (b)  The  main  form  of  dissemination  took  place  in  the  form  of  presentations  at 
international  conferences.  As  well,  a  post-doctoral  fellow  from  Dr.  HSR-1's  lab  took  what  he 
learned  with  him  to  Mexico  where  he  is  involved  in  training  pulmonary  specialists  in  Mexico  and 
South  America.  Through  the  University  where  Dr.  HSR-1  is  a  faculty  member  there  is  an  office 
of  'technology  transfer'  that  provides  business  advice  and  strategy  for  the  commercialization  of 
SNORESAT®. 

Secondary  Outputs  The  post-doctoral  fellow  continues  to  look  into  the  relationship  between 
childhood  disorders  and  obstructive  sleep  apnea.  In  addition,  researchers  in  the  Cardiology 
field  are  looking  further  into  the  relationship  between  sleep  apnea  and  heart  disease.  A 
commercial  company,  after  becoming  familiar  with  Dr.  HSR-1 's  research  on  sleep  apnea, 
commissioned  him  to  do  a  health  surveillance  study.  The  study  addresses  the  relationship 
between  sleep  apnea  and  fatigue-related  accidents. 

Applications  It  is  reported  that  there  has  since  been  some  adoption  of  the  technology  and 
there  is  likely  to  be  a  future  shift  toward  diagnosis  being  performed  by  the  general  practitioner 
that  may  preclude  a  visit  to  a  sleep  specialist.  (A  quantification  of  the  extent  of  usage  could  be 
obtained  from  billing  records). 

Impacts  or  Final  Outcomes  The  main  benefits  to  the  health  service  will  be  in  terms  of  easier 
patient  access  to  diagnosis  and  treatment  and  hence,  more  widespread  treatment  of  patients 
than  before  the  introduction  of  this  technology.  The  cost  per  patient  will  be  considerably  lower, 
with  diagnosis  that  can  be  undertaken  by  general  practitioners  rather  than  specialists.  There  is 
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pending  commercial  development  of  the  SNORESAT®  device.  (In  future,  sales  figures  could  be 
obtained  to  provide  an  indication  of  the  commercial  benefits  in  terms  of  royalties  and  as  an 
indication  of  how  widely  accepted  the  device  is  as  a  diagnostic  tool  for  sleep  apnea). 


28 


Case-Study  2:  Insight  into  back  x-rays 

The  principal  investigator  for  this  study  was  Dr.  HSR-2.  The  researcher  is  an  epidemiologist, 
associate  professor  in  the  University  of  Alberta's  Faculty  of  Medicine  and  Dentistry,  an  Arthritis 
Society  Scholar  and  a  Heritage  Population  Health  Investigator.  The  project  was  a  collaborative 
effort  of  the  U  of  A's  Healthcare  Quality  and  Outcomes  Research  Centre  and  investigators  in  the 
Alberta  Family  Practice  Research  Network. 

Research  Needs  Assessment  Practice  variations,  where  physicians  adopt  different 
approaches  to  the  treatment  of  particular  patient  groups,  are  commonly  found  in  medicine.  Dr. 
HSR-2  felt  that  such  variations  in  the  use  of  X-rays  for  patients  with  lower  back  pain  was  a 
cause  for  concern.  Therefore,  it  was  appropriate  usage  of  radiographs  for  this  indication  that 
was  investigated. 

The  concern  was  radiation  load.  For  women,  back  x-rays  have  a  risk  of  affecting  the 
reproductive  organs.  The  radiation  in  a  set  of  lower  lumbar  x-rays  is  equivalent  to  a  woman 
having  a  daily  chest  x-ray  for  six  years.  As  part  of  the  research  plan  Dr.  HSR-2  retrospectively 
determined  patterns  of  practice  among  family  physicians  in  relation  to  the  use  of  lumbar  spine 
radiography  in  patients  with  low  back  pain.  In  addition,  the  sensitivity,  specificity  and  predictive 
value  of  lumbar  spine  radiographs  in  the  diagnosis  of  low  back  pain  disorders  were  assessed. 

At  the  time  of  this  study  the  U.S.  Agency  for  Health  Care  Policy  and  Research  (AHCPR)  had 
recently  published  guidelines  for  management  of  patients  with  acute  low  back  pain,  which 
included  recommendations  for  the  use  of  lumbar  radiographs.  The  purpose  of  this  retrospective 
study  was  to  evaluate  the  potential  impact  of  these  guidelines  in  patients  with  new  episodes  of 
low  back  pain  seen  by  primary  care  physicians. 

interface  (a)  As  stated  in  Case  Study  1 ,  expert  scientific  review  is  part  of  the  process  used  by 
the  AHFMR  in  deciding  which  research  to  fund.  The  research  proposal  was  developed  as  a 
collaborative  effort  between  the  principal  investigator  (PI),  the  Healthcare  Quality  and  Outcomes 
Research  Centre  at  the  U  of  A,  and  the  Alberta  Family  Practice  Research  Network.  The  Pi's 
involvement  with  the  Cochrane  Collaboration  and  the  Arthritis  Society  provide  opportunities  for 
both  extramural  collaboration  and  further  planning  of  the  research  agenda. 
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Inputs  The  funding  for  this  study  was  received  from  Alberta  Health's  Health  Services  Research 
and  Innovation  Fund  administrated  by  the  AHFMR.  The  amount  was  $1 14,800  over  two  years 
(94/95  and  95/96).  Expertise  of  the  researchers  included  a  clinical  epidemiologist  MD,  MSc, 
PhD  (the  PI)  and  two  co-investigators;  a  family  practitioner  representing  the  Alberta  Family 
Practice  Research  Network  and  the  Head  of  the  Division  of  Rheumatology. 

Processes  The  study  was  3-staged:  1)  a  retrospective  study  of  medical  records  of  963  patients 
who  presented  with  LBP  at  a  family  practice,  2)  a  prospective  study  to  examine  the  outcome  of 
patients  with  LBP,  and  3)  compliance  with  AHCPR  guidelines,  and  the  guidelines  published  by 
Deyo,  were  assessed.  As  a  result  of  reviewing  the  AHCPR  guidelines  for  low  back  pain  it  was 
suggested  that  a  more  restricted  and  cost-efficient  set  of  guidelines  should  be  proposed.  It  was 
intended  that  this  research  would  a)  influence  clinical  practice,  b)  inform  the  researchers 
regarding  methodological  principles,  and  3)  inform  and  influence  policy.  The  ultimate  goal  was 
to  improve  the  health  and  quality  of  care  of  patients. 

Primary  outputs  The  review  of  patient  records  showed  that  1 3%  had  been  given  lumbar 
radiographs.  If  the  recently  released  clinical  guidelines  had  been  applied  to  this  group,  44%  of 
patients  would  have  undergone  radiography.  This,  in  turn,  would  have  increased  utilization  by 
238%,  and  with  it  the  radiation  load.  Abstracts  were  presented  at  the  American  College  of 
Rheumatology  National  Meeting,  two  in  1995,  one  in  1996  and  two  in  1998.  A  journal  article 
describing  the  results  of  part  of  this  research  was  published  in  the  Journal  of  the  American 
Medical  Association  (JAMA),  June  1 1 ,  1 997  Vol.  277,  No.  22.  Additionally,  the  JAMA  abstract 
appeared  in  the  American  Journal  of  Managed  Care. 

Interface  (b)  In  addition  to  the  presentations  at  the  American  College  of  Rheumatology 
National  Meetings,  these  research  findings  were  disseminated  in;  clinicians'  and  investigators' 
newsletters,  and  managed  care  journals.  The  PI  mentioned  that,  typically,  after  the  research 
data  has  been  compiled,  there  is  discussion  with  the  key-players  to  decide  on  how  best  to 
disseminate  the  results.  A  good  quality,  peer-reviewed  journal  may  be  the  best  way  to  influence 
both  policy  and  clinical  behaviour. 

Secondary  outputs  Arthritis/rheumatology  researchers  from  other  centres  in  North  America 
and  overseas  have  expanded  on  the  Pi's  research  by  replicating  some  of  these  studies  and 
produced  similar  results.  There  has  been  a  positive  feedback  to  the  PI  about  this  research 
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project.  Although  there  may  not  be  a  rewriting  of  the  AHCPR's  clinical  practice  guideline  (they 
are  no  longer  writing  CPG's)  there  has  been  a  change  in  clinical  practice  which  is  the  action  that 
rewriting  a  CPG  would,  hopefully,  effect.  Worker's  Compensation  organizations  have  shown 
interest  in  the  kinds  of  research  the  PI  is  involved  with  which,  in  turn,  may  result  in  another  form 
of  policy  change. 

Applications  Family  practitioners  involved  in  the  study  were  informed  early  about  the  radiation 
load  involved  if  AHCPR  guidelines  were  used  in  the  diagnosis  of  low  back  disorders  and  may, 
as  a  result,  be  more  selective  in  the  use  of  lumbar  radiographs  than  the  AHCPR  guidelines 
would  have  recommended. 

Impacts  or  final  outcomes  In  the  context  of  this  research  project,  the  impacts  and  final 
outcomes  would  be  identified  as  cost  savings,  increased  efficiency  and  improved  quality  of 
patient  care.  If  the  AHCPR  guidelines  are  to  be  used  in  the  treatment  of  patients  with  low  back 
pain,  there  may  be  unnecessary  increases  in  utilization  and  economic  costs. 
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HTA  Unit  Products 


The  HTA  Unit  of  the  AHFMR  produces  HTA  Reports  and  HTA  Briefs.  Reports  in  these  series 
are  peer-reviewed  prior  to  publication.  As  well,  the  Unit  provides  rapid  assessments  referred  to 
as  Technotes,  which  are  not  subject  to  external  review.  The  HTA  products  are  intended  to  be 
unbiased,  transparent  documents. 

Alberta  Health,  RHAs  and  other  organizations  use  information  in  the  HTA  products  as  an 
adjunct  to  other  information  in  order  to  more  effectively  set  policy.  Requests  for  assessments 
are  prioritized  by  the  Unit  taking  account  of  their  origin,  urgency  and  timelines  involved  to 
undertake  the  assessment,  and  the  level  of  analysis  required.  The  time  required  to  prepare  an 
externally  peer-reviewed  report  varies,  but  is  typically  of  the  order  of  6  months.  Non-externally 
reviewed  reports  (Technotes)  can  take  2  months. 

Once  the  type  of  report  and  the  question  to  be  answered  is  defined,  a  researcher  from  the  HTA 
Unit  is  assigned  to  the  project  and  a  project  outline  agreed  upon.  A  literature  review  on  the 
subject  is  undertaken,  and,  if  pertinent  to  the  report,  additional  information  such  as 
administrative  data  will  be  obtained.  Depending  on  the  type  of  document  being  prepared, 
interviews  with  experts  and  other  pertinent  parties  will  be  arranged.  A  synthesis  is  then  made  of 
available  data. 
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Case-Study  3(a):  Stereotactic  radiosurgery:  options  for  Albertans 

Research  Needs  Assessment  The  request  for  an  assessment  of  stereotactic  radiosurgery 
(SRS)  came  from  policy  makers  within  Alberta  Health.  Patients  were  being  sent  outside  the 
province  for  SRS  therapy.  The  principal  question  was  whether  there  were  any  differences  in 
efficacy  of  two  approaches  to  SRS;  the  Gamma  Knife®(GK)  and  the  modified  linear  accelerator 
(LINAC).  There  was  also  interest  in  the  efficacy,  effectiveness  and  safety  of  SRS  in  the 
management  of  several  conditions. 

Interface  (a)  The  HTA  Unit,  in  consultation  with  the  policy  makers  decided  that  a  peer-reviewed 
report  would  be  prepared.  The  researcher  assigned  to  the  project  developed  the  shape  and 
content  of  the  assessment.  During  the  preparation  of  this  report  there  was  continued  interaction 
between  the  HTA  Unit  and  Alberta  Health.  The  scope  of  the  literature  review  was  refined  as  a 
result  of  these  interactions.  As  well,  the  report  would  include  a  determination  of  the  costs  of 
different  options  for  SRS  and  some  consideration  of  quality  assurance  issues. 

inputs  The  project  was  supported  financially  through  the  HTA  program.  Input  for  the 
assessment  included  advice  from  experts  in  the  fields  of  radiation  oncology,  medical  physics, 
neurosurgery,  and  health  economics.  The  time  invested  in  compiling  the  report  spanned  5 
months.  The  number  of  full-time  equivalents  would  be  3.5  months.  The  persons  involved  in 
preparing  the  report  were  the  Research  Associate  (BSc,  MSc,  major  time  commitment),  the 
AHFMR  librarian,  (BSc,  MLS),  Director  of  the  HTA  Unit,  (PhD),  a  Health  Economist,  (PhD),  from 
the  University  of  Alberta  and  the  HTA  Administrative  Assistant. 

Processes  The  information  for  the  SRS  report  built  on  information  in  previous  HTAs 
undertaken  by  agencies  in  several  countries.  The  literature  review  was  updated  from  the  time 
of  the  last  of  these  assessments.  As  well,  interviews  with  experts  in  the  field  of  SRS,  were 
undertaken.  During  the  preparation  of  HTA  9  two  further  requests  for  information  on  SRS  were 
received  from  Alberta  Health.  A  Technote  and  an  Information  Paper  were  prepared  in  response 
to  these  requests  reflecting  an  ongoing  interaction  between  the  process  of  research  needs 
assessment  by  the  policy-makers  and  the  research  process  itself.  The  draft  report  was 
reviewed  and  modified  internally  and  then  sent  to  external  reviewers  (a  Radiation  Oncologist 
from  the  Cross  Cancer  Institute,  a  Medical  Physicist  from  the  Tom  Baker  Cancer  Centre  and  a 
Radiation  Oncologist  from  McGill  University,  Montreal).  The  draft  was  also  delivered  in  person 
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to  Alberta  Health  so  that  some  discussion  could  occur.  Final  editing  of  the  report  took  account 
of  comments  received. 

Primary  outputs  The  final  report,  HTA  9,  was  published  by  the  AHFMR  in  March,  1998  (32). 
The  report  confirmed  findings  from  other  assessments  that: 

•  the  quality  of  the  available  evidence  on  SRS  effectiveness  is  limited. 

•  there  is  insufficient  information  to  determine  the  comparative 
effectiveness  of  GK  versus  LINAC. 

•  the  GK  approach  is  more  expensive  than  the  LINAC. 

•  surgery  for  the  indications  considered  remains  the  treatment  of  choice 
in  most  cases.  SRS  has  a  role  in  selected  cases  when  surgery 

is  not  an  option. 

•  as  there  is  no  known  difference  in  efficacy  between  the  GK  and  LINAC, 
and  the  former  is  more  expensive,  referral  of  patients  for  treatment  by  GK 
is  not  appropriate. 

The  potential  caseload  for  Alberta  is  uncertain  but  estimated  to  be  30  to  50  per  year.  An 
Alberta-based  SRS  facility  might  cost  about  $4,000  per  case,  at  a  caseload  of  30  per  year.  This 
would  be  substantially  lower  than  referral  of  patients  to  other  provinces  or  the  USA. 

Interface  (b)  The  dissemination  of  HTA  reports  takes  place  by  providing  copies  to  key-players. 
In  the  case  of  HTA  9  the  key-players  were  policy-makers  within  Alberta  Health,  medical 
physicists,  neurosurgeons,  radiation  oncologists.  Regional  Health  Authorities,  the  College  of 
Physicians  and  Surgeons  and  the  Alberta  Medical  Association.  A  structured  abstract  appears 
on  the  AHFMR  website,  which  allows  the  option  of  downloading  the  full  report.  Reports  were 
also  sent  out  to  all  that  contributed  information  and  in  response  to  requests.  The  report  is  also 
included  in  the  registry  maintained  by  the  International  Network  of  Agencies  for  Health 
Technology  Assessment  (INAHTA)  Secretariat  and  the  DARE  database. 

Secondary  outputs  Alberta  Health  was  contacted  to  determine  the  utilization  of  the  SRS 
report.  They  reported  that  the  Out-of-Country  Health  Services  Committee  has  used  it 
extensively.  Since  March  1998  they  have  received  several  requests  for  Albertans  to  travel  to 
the  USA  to  obtain  SRS  using  the  Gamma  Knife®  technique.  These  have  been  denied  and 
patients  have  gone  to  Ontario  for  treatment  or  will  be  going  in  the  near  future.  The  committee 
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has  sent  a  letter  advising  the  Minister  of  Health  that  based  on  this  report,  they  will  not  be 
approving  requests  for  patients  to  attend  the  USA  facilities  for  SRS  and  have  asked  for  his 
support  in  this  decision. 

Applications  The  report  has  (possibly)  provided  background  and  support  for  subsequent 
decisions  by  the  Alberta  Cancer  Board  to  acquire  SRS  technology  in  Calgary. 

Impact/Final  Outcome  Impacts  and  final  outcomes  are  identified  as  cost-savings  to  Alberta 
Health  Care,  more  appropriate  referral  and  allocation  of  certain  patients  to  treatment  options. 
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Case-Study  3(b):  Telephone  nurse  triage  services 

Research  Needs  Assessment  The  request  for  this  Technote  came  from  a  Regional  Health 
Authority  (RHA).  They  asked  that  an  evaluation  be  undertaken  of  the  literature  regarding  the 
implementation  of  a  telephone  nurse  triage  service  that  answers  calls  and  carefully  assesses 
the  caller's  symptoms.  This  typically  involves  the  use  of  either  computerized  or  written  protocols 
that  incorporate  an  algorithm  or  'decision  tree'.  Callers  are  given  advice  based  on  their 
symptoms.  This  may  include  self-care  at  home  or  being  directed  to  the  most  appropriate  health 
care  setting  (4).  The  nurse  triage  service  might  provide  an  answer  to  the  growing  demands 
being  placed  on  emergency  medical  services  in  Alberta.  The  triage  service  would  handle  non- 
urgent health-care  issues,  which  would,  in  theory,  reduce  the  number  of  emergency  department 
visits. 

Interface  (a)  During  the  preparation  of  the  Technote  there  was  contact  with  the  RHA.  A 
preliminary  outline  of  the  report  was  sent  to  them  for  their  comments.  They,  in  turn,  suggested 
additional  information  to  include  in  the  report  and  the  outline  was  revised  accordingly. 

Inputs  The  time  invested  in  compiling  the  Technote  was  spread  over  2  months  and  equivalent 
to  about  2  weeks  full  time  research.  The  persons  involved  in  preparing  the  report  were  the  HTA 
researcher,  the  AHFMR  libranan.  Director  of  the  HTA  Unit,  and  the  HTA  Administrative 
Assistant. 

Processes  Preparation  of  a  such  a  Technote  involves  a  expeditious  survey  of  available 
literature.  Additional  information  is  sought  through  interviews,  in  this  case,  with  a  Canadian 
service  that  provides  telephone  nurse  triage.  After  a  review  of  available  literature  a  draft 
Technote  was  prepared,  reviewed  by  the  Director  of  the  HTA  Unit,  edited  and  sent  to  the  RHA. 

Primary  outputs  The  Technote  was  published  by  the  AHFMR  in  April,  1998.  It  stated  that  if  a 
telephone  nurse  triage  service  were  to  be  implemented  in  Alberta  there  would  be  a  need  to  look 
at:  the  population  to  be  serviced,  protocols  to  be  used,  staffing  needs/training,  and  any  direct 
links  to  other  services. 

It  was  concluded  that: 

•        there  are  several  telephone  nurse  triage  services  in  the  U.S.,  one  in 
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Canada; 

•  there  is  evidence  that  telephone  nurse  triage  services  are  useful,  but  this 
is  based  on  poor  quality  studies; 

•  data  on  outcomes  are  lacking  in  terms  of  use  of  services  and 
health  status  of  callers; 

•  legal,  organizational,  and  economic  matters  need  to  be  resolved. 

Good  quality  studies  are  required  to  more  firmly  establish  the  health-care  and  monetary  benefits 
of  a  telephone  nurse  triage  service. 

Interface  (b)  The  dissemination  of  HTA  Technotes  takes  place  by  supplying  copies  to  key- 
players  in  Alberta.  In  the  case  of  TN14  these  included  the  Regional  Health  Authority  that 
requested  the  information,  other  RHA's,  Alberta  Health,  the  Alberta  Assocation  of  Registered 
Nurses  and  others.  Titles  of  Technotes  are  listed  on  the  AHFMR  website,  and  can  easily  be 
requested  on-line  or  by  phone.  Technotes  are  provided  free  of  charge.  Several  requests  have 
been  received  from  out-of-province. 

Secondary  outputs  Calgary  RHA  has  used  the  report  as  input  to  further  appraisal  of  a 
proposed  service.  Alberta  Health  has  reviewed  the  report  and  followed  up  by  asking  the  HTA 
unit  for  more  information  about  telephone  triage  nursing  services.  AH  plans  to  conduct  a 
feasibility  study  regarding  the  implementation  of  such  a  service  in  the  Province. 

Applications  The  Calgary  Regional  Health  Authority  (CRHA)  was  contacted  with  respect  to 
their  interest  in  a  telephone  nurse  triage  service.  The  final  report  of  a  feasibility  study  conducted 
by  the  CRHA  was  completed  late  this  year  (1998).  They  addressed  accessibility  to  the  service, 
training  of  personnel  and  costs.  The  report  has  now  gone  to  their  Management  Committee, 
along  with  the  Technote,  as  part  of  an  information  package  to  be  used  for  further  decision- 
making. 

Impact/Final  Outcome  Information  on  impact  and  final  outcome  of  telephone  nurse  triage 
services  is  not  available  at  this  time. 

Conclusions  from  HSR  Case  Studies 

Even  from  the  limited  amount  of  work  put  into  these  case  studies,  it  is  quite  clear  that,  as 
expected,  the  Buxton/Hanney  approach  works  in  a  comparable  way  in  relation  to  these  AHFMR 
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HSR  projects  as  to  the  UK  DH  projects  for  which  it  was  developed.  The  case  studies  raised  no 
new  problems  or  specific  difficulties.  Much  of  the  work  on  these  case  studies  was  undertaken 
by  WLS,  who  was  thus  far  unfamiliar  with  the  approach.  This  acted  as  a  reminder  that,  while 
others  can  readily  use  the  approach,  there  is  a  learning  process  to  gain  familiarity  with  the 
categorization  and  stages  involved.  The  case  studies  also  remind  us  that  the  framework  is  not 
rigid,  but  needs  to  be  used  flexibly  to  accommodate  the  circumstances  of  particular  projects. 
For  example,  as  previously  noted,  interaction  with  funders  and  stakeholders  may  continue 
throughout  the  project,  and  the  process  of  project  specification  may,  as  in  some  of  the  HTA 
examples,  be  an  ongoing  process  throughout  the  project. 

These  case  studies  show  that  it  is  possible  to  go  a  considerable  way  to  analyze  a  project  in 
terms  of  the  model  using  existing  'on  file'  sources  of  information,  but  that  these  typically  leave 
some  important  gaps.  The  implication  of  this  is  that,  if  the  AHFMR  were  to  decide  to  undertake 
more  routinely  such  ex  post  analyses  of  its  funded  research  this  could  be  facilitated  by  making 
researchers  aware  of  this  and  by  requesting  that  certain  relevant  information  be  clearly  provided 
in  annual  or  end  of  project  reports.  In  the  case  of  the  latter,  however,  some  of  the  information 
would  not  be  available  immediately  at  the  end  of  the  project,  and  there  may  be  an  argument  for 
an  'after  project'  review  report  say  two  years  after  the  end  of  the  relevant  funding.  While  this 
might  be  seen  as  an  unnecessary  and  undesirable  additional  burden  by  some  researchers,  for 
those  receiving,  and  expecting  to  receive,  ongoing  support  from  the  AHFMR  it  might  be 
recognized  as  a  reasonable  request.  Attitudes  may  depend  on  the  general  climate  as  to 
whether  it  seems  that  such  evidence  is  important  in  the  ongoing  process  of  justifying  and 
funding  research,  and  the  observed  policy  of  the  AHFMR  will  be  the  main  determinant  of  this 
climate  of  opinion  locally. 
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SECTION  5: 


EXTENDING  THE  MODEL  TOWARDS  CLINICAL  AND  BASIC  BIOMEDICAL 
RESEARCH 

Introduction 

For  the  most  part,  research  to  date  has  applied  the  Buxton/Hanney  approach  to  examples  of 
applied  health  services  research,  particularly  evaluation  of  particular  forms  of  health  care.  This 
work  has  already  noted  that  in  the  case  of  more  methodologically  orientated  research,  the 
health  service  benefit,  if  any,  will  typically  arise  through  subsequent  use  of  the  methods  rather 
than  directly  from  the  methodological  research  project  itself.  By  implication,  the  same  argument 
might  apply  to  more  basic  research  in  health  sciences,  where  health  service  (and  similar 
benefits)  arise  as  a  result  of  the  incorporation  of  the  results  of  the  basic  research  into  more 
applied  research.  Indeed  there  might  be  a  series  of  'feeds'  into  progressively  more  applied  and 
service  focused  research,  before  health  service  benefits  arise. 

This  section  considers  some  of  these  possibilities  in  the  light  of  previous  attempts  to  analyze  the 
role  and  value  of  basic  science.  It  then  uses  four  case  studies  to  begin  to  see  how  the  existing 
model,  or  extensions  of  it,  might  be  used  to  analyze  the  research  of  clinical  investigators  and 
basic  scientists.  It  then  draws  some  tentative  conclusions,  recognizing  that  the  work  here 
represents  no  more  than  a  first  foray  into  a  very  complex  field  of  study. 

Justifying  basic  science 

The  need  for  researchers  to  justify  spending  on  basic  science  seems  to  arise  in  regular  cycles. 
Most  of  the  best  known  analyses  of  the  'value'  of  basic  science  have  been  carried  out  as 
responses  to  perceived  threats  to  future  funding  or  to  scientists'  freedom  to  pursue  pure 
curiosity-led  rather  than  objective-focused  research. 

One  of  the  first  studies  tracing  the  role  of  basic  research  in  contributing  to  advances  of  social 
and  economic  importance  looked  at  oral  contraception  (29).  It  pointed  to  the  role  of  the 
convergence  of  a  number  of  different  streams  of  scientific  knowledge  -  the  physiology  of 
reproduction,  hormones  and  their  functioning  and  steroid  chemistry  that  was  necessary  for  the 
development  of  an  oral  contraceptive.  The  study  importantly  established  the  tracing 
methodology  as  a  way  of  portraying  research  advance. 
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A  more  recently  cited  study  using  the  same  basic  approach  is  that  of  Comroe  and  Dripps  (13). 
This  major  study  was  undertaken  as  a  response  to  the  Johnson  Administration  in  the  USA  in 
1966  that  expressed  concern  that  too  much  basic  science  remained  'locked  up  in  the 
laboratory',  and  that  the  lag  between  initial  discovery  and  final  application  was  over  long.  It  was 
believed  that  this  situation  could  be  improved  if  basic  science  were  made  more  'mission- 
orientated'  or  in  the  case  of  medical  research  more  'disease-orientated'.  The  result  was  an 
upsurge  in  contract  research  and  commission-initiated  research  supported  by  the  National 
Institutes  of  Health. 

This  position  was  justified  in  part  by  the  conclusions  of  a  preliminary  report  of  a  study,  Project 
Hindsight,  commissioned  by  the  Department  of  Defense  and  published  in  1966  (34).  This  study 
analyzed  ex  post  how  twenty  important  military  weapons  came  to  be  developed.  The 
conclusions  were  that:  the  contributions  of  university  research  were  minimal;  scientists 
contributed  more  effectively  when  their  effort  was  mission-orientated,  and  the  lag  between  initial 
discovery  and  final  application  was  shortest  when  the  scientists  worked  in  areas  targeted  by 
their  sponsor. 

Comroe  and  Dripps  argued  that  scientists  had  to  counter  these  arguments  with  more  than 
illustrative,  highly  selective,  and  largely  anecdotal  case  studies.  Therefore,  they  set  up  a  study 
of  a  whole  field  -  cardiovascular  and  pulmonary  diseases,  and,  used  40  physicians  to  identify 
the  top  ten  clinical  advances  in  the  previous  30  years.  Then  with  140  consultants  they  identified 
the  essential  bodies  of  knowledge  necessary  to  achieve  the  advances.  Thus  for  example,  they 
identified  the  sequence  of  37  key  scientific  events  that  necessarily  preceded  the  development  of 
electrocardiography:  a  chronology  stretching  back  to  the  scientific  advances  of  ancient 
civilizations,  through  to  a  number  of  final  key  steps  in  the  post-war  period.  For  each  step,  where 
possible,  they  identified  a  key  article  that  was  most  significant  in  encapsulating  and  reflecting 
the  element  of  scientific  advance.  Analysis  of  these  529  key  articles  led  them  to  conclude  that: 

•  41  %  of  all  work  judged  to  be  essential  for  clinical  advance  was  not  clinically 
orientated  at  the  time  it  was  done  -  the  scientists  responsible  for  these  articles 
sought  knowledge  for  the  sake  of  knowledge; 

•  62%  of  these  articles  described  basic  rather  than  applied  research; 

•  clinical  advances  require  a  combination  of  different  types  of  research  and 
development,  and  not  one  to  the  exclusion  of  another. 
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This  classic  study  emphasizes  the  complexity  of  the  science  base  for  many  current  advances, 
the  diversity  of  component  elements  on  which  they  drew,  and  the  long  time  lags  between  some 
scientific  advances  and  their  useful  application. 

More  recently  in  1993,  seemingly  in  response  to  more  recent  threats  to  basic  science,  Raiten 
and  Berman  published  an  analysis  of  the  development  of  monoclonal  antibodies  that  traced  the 
seminal  scientific  events  from  the  time  of  Pasteur  and  Koch  in  1876,  onwards  (31).  They  went 
on  to  estimate  the  very  favourable  returns  on  investment  involved  in  this  case.  However,  their 
analysis  suffers  from  the  fatal  logical  flaw  of  only  allowing  for  the  costs  of  the  science  research 
that  subsequently  proved  useful  and  not  all  the  research  that  was  funded  but  may  not  have  had 
any  subsequent  importance.  (It  is  'epidemiology'  without  a  population  base!) 

In  each  of  these  studies  cited,  the  path  was  traced  ex  posf  from  application  back  to  basic 
science.  It  should,  in  principle,  be  possible  ex  post  to  trace  the  path  in  the  opposite  direction 
from  a  single  major  piece  of  basic  science  research  to  the  various  applications  that  have  directly 
or  indirectly  drawn  upon  it;  what  has  been  referred  to  as  the  'knowledge  cycle'  (30).  To 
understand  the  eventual  payback  from  particular  investments  in  basic  science  it  would  be 
necessary  to  trace  the  linkages  fonA/ard  from  a  single  piece  or  program  of  research  as  they  are 
perhaps  taken  up,  or  influenced,  by  an  increasingly  disparate  range  of  subsequent  research  and 
eventual  applications.  In  principle  this  could  link  basic  science  to  the  specific  health  benefits 
which  have  later  resulted  at  least  in  part  from  it.  This  work  has  been  facilitated  by  the 
development  of  citation  indices,  pioneered  by  Eugene  Garfield  at  the  Institute  for  Scientific 
Information  (18).  However  the  data  from  them  needs  to  be  interpreted  with  caution  (19). 

There  are  a  number  of  implications  of  these  complex  linkages  between  basic  science  and 
application.  The  first  is  that  the  time-scales  can  be  very  long  indeed  and  important  discoveries 
may  lie  dormant  for  many  years  until,  with  the  juxtaposition  of  another  advance,  they  are  taken 
up  and  applied  (14).  Although  some  basic  research  ideas  may  be  taken  up  and  applied  rapidly, 
generally  such  assessments  of  research  would  need  to  take  place  after  a  significant  time  has 
elapsed.  The  second  implication  is  that  basic  science  may  have  very  disparate  and 
unforeseeable  impacts.  This  means  that  the  process  of  attempting  to  identify  and  describe  such 
impacts  may  involve  a  process  of  search  that  goes  very  wide  indeed,  as  distinct  from  the  very 
focused  search  that  might  follow-up  on  a  specific  piece  of  applied  research. 
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An  added  complexity  is  that  frequently  knowledge  develops  through  accretion  and  rather  than 
the  results  of  a  particular  study  leading  directly  to  an  application,  the  results  accumulate  in  a 
reservoir  of  knowledge  (38,39). 

An  alternative  approach  may  be  simply  to  accept  that  for  much  basic  science  the  knowledge 
generation  is  the  only  measurable  payback.  But  this  begs  a  question  as  to  how  one  would 
value  that  knowledge.  Ex  post,  and  with  20/20  hindsight,  many  would  probably  value  a 
particular  piece  of  scientific  knowledge  in  terms  of  the  importance  of  that  knowledge  when 
subsequently  applied.  Certainly,  if  the  objectives  in  funding  that  research  are  the  pursuit  of 
human  health  rather  than  simply  knowledge,  then  relevance  as  identified  by  valuable  application 
would  appear  to  be  a  more  appropriate  measure  of  payback  than  simply  some  measure  of 
'volume'  of  knowledge  generated. 

Experience  with  the  case  studies 

With  such  issues  in  mind,  four  case  studies  of  important  examples  of  long-term  research 
support  provided  by  the  AHFMR  were  used  as  vehicles  to  explore  the  relevance  of  the  model, 
and  how  it  might  be  extended  or  elaborated  to  deal  with  more  basic  research.  Two  case  studies 
related  to  researchers  who  were  clinician  investigators  and  two  related  to  researchers  who  were 
basic  biomedical  scientists.  Below  is  a  summary  of  each  case  study  using  the  model's  format. 
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Case-Study  4:  Transplantation  of  pancreatic  islets  of  Langerhans 

Research  Needs  Assessment  Diabetes  mellitus  is  a  medical,  financial  and  social  'time  bomb' 
to  thousands  of  Canadians  each  year.  It  causes  blindness,  kidney  failure,  stroke,  gangrene  and 
impotence.  Normalizing  circulating  glucose  throughout  the  day  can  prevent  these  serious 
complications.  An  attractive  approach  is  islet  cell  transplantation  that  has  been  shown  to 
succeed  in  several  Type  I  diabetic  patients  throughout  the  world.  The  aims  of  this  research  are 
to  develop  transplantation  of  insulin-producing  islets  by  reducing  antirejection  drugs  for 
recipients,  enhancing  the  supply  of  islets  for  transplantation,  and  storage  of  the  islets  in  an  ultra- 
low  frozen  state  (cryopreservation). 

As  a  clinician.  Dr.  A  thought  this  would  be  his  opportunity  to  bring  science  to  the  bedside.  As 
well,  he  wanted  to  satisfy  his  curiosity,  liked  the  complexity  and  wanted  to  contribute  to  the 
solution  of  the  problem  of  Type  I  diabetes  as  well  as  extend  the  availability  of  this  treatment  to 
Type  II  diabetic  patients.  Ideally,  this  would  be  before  advanced  complications  set  in.  [The 
bigger  'need',  of  course,  would  be  coming  from  the  patients  who  are  unable  to  produce  sufficient 
insulin.] 

Interface  As  previously  stated,  expert  scientific  review  is  part  of  the  process  used  by  the 
AHFMR  in  deciding  which  research  to  fund.  An  'interface'  would  be  the  exchange  of  ideas  from 
the  reviewers  to  Dr.  A,  which,  in  turn,  would  have  seen  him  revise  his  proposal.  This  particular 
project  builds  on  Dr.  A's  successes  demonstrated  in  1990  for  islet  cell  transplantation. 
Reviewers  of  this  proposal  were  strongly  in  favour  of  the  project.  One  reviewer  suggested  that 
Dr.  A  look  into  ways  of  developing  a  technique  for  islet  transplant  that  does  not  require  any  form 
of  immunosuppression.  The  interface  of  Dr.  A  with  other  researchers  is  mainly  through  DRINET 
(Diabetes  Research  Institute  Network).  It  was  set  up  4  years  ago  with  the  goal  of,  among  other 
things,  steering  around  competitiveness,  promoting  more  collaborative  efforts,  sharing  protocols 
and  realizing  that  it  takes  a  global  perspective  to  solve  the  problem  of  treating/curing  Type  1 
diabetes.  There  are  8  other  groups  in  DRINET:  two  in  the  UK,  one  in  Germany,  one  in  Sweden, 
and  four  in  USA. 

Inputs  Dr.  A  and  his  team  received  grant  support  totaling  around  $380,000  in  1996/97. 
Sources  included  The  Alberta  Foundation  for  Diabetes  Research,  Capital  Health  Authority, 
Health  Services  Research  and  Innova,  Jdf  International,  Medical  Research  Council  of  Canada, 
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and  Universities  Hospitals  Foundation.  Dr.  A  is  an  AHFMR  Senior  Medical  Scholar  (1996- 
2001).  He  is  recognized  nationally  and  internationally  for  his  outstanding  contributions  to  the 
field  of  islet  transplantation.  He  and  his  associates  were  the  first  to  demonstrate  long  term 
function  of  transplanted  islets  into  a  Type  1  diabetic  patient,  in  addition,  they  demonstrated  that 
it  was  possible  to  use  cryopreserved  islets  for  the  transplants  and  thus  utilize  human  islets  from 
several  different  pancreases.  He  holds  a  medical  degree  (MD)  from  the  University  of  Alberta, 
and  an  MSc  (experimental  surgery)  from  the  University  of  Alberta.  He  is  now  a  Professor  and 
Director  of  Surgery,  in  the  Division  of  Surgical  Research. 

Processes  The  two  major  research  objectives  were  1)  to  determine  the  minimum  quantity  of 
human  islet  tissue  needed  to  induce  euglycemia  after  transplantation;  and  2)  to  determine 
whether  immunomodulation  of  islets  using  tissue  culture,  cryopreservation,  or  gene  therapy  will 
reduce  their  susceptibility  to  immune  injury. 

Primary  outputs  Journals  that  have  published  Dr.  A's  work  (1996)  include:  American  Journal 
of  Surgery  (1  article).  Transplantation  (3  articles),  Xenotransplantation  (1  article),  Journal  of 
Clinical  Investigation  (1  article),  and  Cell  Transplantation  (1  article). 

Key  findings  include  the  development  of  ex  vivo  gene  therapy  of  intact  islets  that  protects  islets 
from  rejection  after  transplantation.  Improved  recovery  of  human  islets  for  transplantation  has 
been  possible  through  the  identification  of  variables  that  influence  the  isolation  process.  A  new 
bulk  preservation  method  has  been  developed  to  preserve  islets  in  the  ultra-low  frozen  state. 

Dr.  A  also  supervises  a  number  of  students  in  summer  research  programs,  undergraduate 
degree  projects  and  graduate  programs.  He  is  a  major  supervisor  for  an  MD  and  a  PhD 
student.  As  well,  he  is  a  committee  member  for  4  MSc  and  4  PhD  students. 

Interface  (b)  Dissemination  took  place  at  conferences  that  Dr.  A  and  his  collaborators  have 
attended.  These  include  the  7^^  Ottawa  International  Conference  on  Medical  Education  and 
Assessment  (Netherlands),  3^*^  International  Congress  of  the  Cell  Transplant  Society  (Florida), 
65*^  Annual  Meeting  of  the  RCPS  of  Canada  (Nova  Scotia),  International  Symposium  on 
Endocrine  Cell  Transplantation  and  Genetic  Engineering  (Germany).  Other  presentations 
include:  city-wide  endocrine  rounds  (University  of  Calgary),  CDA  43^^  National  Conference 
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(Jasper),  Toronto  Diabetes  Association  Annual  Meeting  and  CDA  "75  Years  of  Progress  in 
Diabetes  Care,  Management  and  Treatment"  (Toronto). 

Dr.  A  is  also  Chairman  of  the  "Research  Development  Committee"  of  the  Canadian  Association 
of  Surgical  Chairmen.  It  is  in  this  environment  that  results  of  his  research  would  be  further 
disseminated. 

Secondary  outputs  Dr.  A  states  that  the  immediate  results  of  his  research  are  being  used  by 
practitioners  treating  Type  I  diabetics.  However,  we  were  unable  to  determine  the  extent  of  this. 
Dr.  A  states  that  some  of  his  research  areas  are  'generalizable',  that  it  to  say  the  methods  are 
being  picked  up  by  others.  An  example  is  researchers  involved  in  cell  replacement.  The  areas 
of  cell  transplant  research  include  muscular  dystrophy,  advanced  myopathies,  neural  cells  for 
Parkinson's  Disease  and  parathyroid  replacements. 

Other  secondary  outputs  would  include  the  rewriting  of  guidelines  in  the  treatment  of  Type  I 
diabetes.  Especially  important,  would  be  the  adoption  of  these  guidelines  by  a  professional 
body.  This  research  provides  an  opportunity  for  intellectual  property  rights. 

Applications  This  information  is  not  available.  However,  an  example  of  'application'  of  this 
work  would  be  if  practitioners  were  using  the  results  of  Dr.  A's  research  to  change  their  practice 
patterns. 

Impact/Final  Outcome  The  timelines  involved  in  measuring  outcomes  of  this  research  are 
indeterminate.  However,  what  one  would  look  for  is  improved  quality  patient  care.  Is  this 
treatment  better  than  anything  offered  currently?  How  is  it  better?  Is  it  more  or  less  costly?  Is 
there  an  increase  in  survival  and/or  quality  of  life  of  patients? 
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Case-Study  5:  Ligament  healing 

Research  Needs  Assessment  The  impetus  for  Dr.  B  to  begin  researcin  in  the  area  of 
orthopaedics  came  from  a  personal  experience  with  knee  surgery.  As  part  of  a  research  project 
he  conducted  a  retrospective  review  of  persons  having  knee  surgery,  specifically  ligament 
surgery.  Many  were  developing  arthritis.  He  realized  the  need  for  further  research  into  the  area 
of  ligament  healing. 

Currently  the  state-of-the-art  replacement  of  damaged  ligaments  is  to  use  autogenous  (host) 
tendon  tissue.  Another  alternative  is  the  use  of  allograft  tissue  (various  tendons  or  ligaments). 
There  is  a  potential  for  all  of  these  tissues,  both  autografts  and  allografts,  to  be  successful. 
However,  they  clearly  require  optimization.  Variables  that  affect  their  rates  and  endpoints  of 
healing  are  the  subject  of  this  research. 

Interface  As  stated  previously,  expert  scientific  review  is  part  of  the  process  used  by  the 
AHFMR  in  deciding  which  research  to  fund.  Any  changes  to  Dr.  B's  proposal  that  had  taken 
place  as  a  result  of  reviewer's  comments  would  be  an  example  of  'interface'  mentioned  in  the 
model.  Dr.  B  collaborates  with  a  biochemist/molecular  biologist  and  an  engineer.  There  is 
sharing  of  ideas  that  comes  mostly  at  conferences/meetings.  A  group  that  Dr.  B  is  part  of  is  the 
"Rocky  Mountain  Ligament  Group".  This  group,  mostly  American,  work  mainly  on  ligaments. 
The  ideas  that  are  shared  amongst  this  group  work  well  in  planning  further  research. 

Inputs  Grants  and  awards  given  to  Dr.  B  include  the  MRC  of  Canada  who  has  given  an 
operating  grant  (94-97)  and  for  the  "Quantification  of  loading  within  normal  and  healing 
ligaments"  project  (97-99).  Another  award  from  the  MRC  was  an  operating  grant  for  95  to  98  for 
"ligament  transplantation".  Awards  from  the  Arthritis  Society  (94-2000)  have  supported  work  on 
"The  effects  of  ligament  contact  on  ligament  healing".  An  operating  grant  was  received  from  the 
Arthritis  Society  (97-2000)  for  "Targeting  of  collagens  by  gene  therapy  to  enhance  ligament 
healing".  The  London  Life  Award  was  given  as  an  operating  grant  for  3  years  for  "Gene  therapy 
to  optimize  ligament  healing". 

Dr.  B  is  an  extremely  well  trained  scientist  and  orthopaedic  surgeon.  He  has  a  MD  degree  and 
did  surgical  and  orthopaedic  training.  He  then  went  on  to  do  significant  research  at  the 
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University  of  California,  San  Diego.  His  multidisciplinary  team  consists  of  researchers  from 
Bioengineering,  Biochemistry,  Morphology  as  well  as  a  collagen  specialist. 

Processes  The  experimental  design  will  build  on  prior  work  in  this  area.  Specifically,  the 
researchers  propose  to  continue  studying  the  potential  for  longer  term  incorporation  or 
replacement  of  grafts  at  insertions  by  examining  bone-ligament-bone  graft  properties  at  the 
longest  practical  (2  and  3  year)  healing  intervals.  They  plan  to  look  at  alternate  grafting 
approaches.  Further,  they  propose  to  use  the  tendon  graft  as  an  experimental  and  theoretical 
model  for  the  study  of  graft  differentiation  under  different  stress  conditions.  Finally,  the 
application  of  existing  methods  and  development  of  new  methods  to  begin  complimentary 
investigations  of  anterior  cruciate  ligaments  (ACLs)  in  a  larger  animal  model  (sheep)  will  be 
attempted.  Methods  include: 

•  Multidisciplinary  assessments  of  both  soft  tissue  and  insertional  portions  of  each  type  of 
graft  at  each  healing  interval. 

•  Biomechanical  measurements  of  both  prefailure  and  failure  properties  will  continue  to  be  the 
main  measures  of  tissue  function. 

•  Morphologic,  histomorphometric  (light  and  electron  microscopic)  and  biochemical  measures 
will  be  used  to  understand  the  processes  of  healing,  reasons  for  graft  differences  and  help 
to  explain  the  mechanical  outcomes. 

•  Finite  element  methods  will  be  used  for  correlative  studies  of  tissue  stresses  versus  cellular, 
mechanical  and  biochemical  differentiation  along  graft  lengths. 

Primary  outputs  There  have  been  16  publications,  1  patent,  16  abstracts  and  4  book  chapters 
coming  out  of  this  research.  The  London  Life  Research  Award  (1997)  was  given  to  Dr.  B  for  his 
work  entitled:  "Gene  Therapy  for  Optimizing  Ligament  Healing".  As  well.  Dr.  B's  (and 
colleagues)  work  in  expanding  the  research  team  at  a  Centre  that  has  attracted  researchers 
from  all  over  the  world. 

Key  findings  of  the  research  include  the  development  and  implementation  of  a  model  and 
methods  for  studying  a  liposome-mediated  gene  transfer  technique  in  ligament  scars.  As  well, 
new  models  and  methods  for  the  investigation  of  ACL  healing,  transplantation  and  repair  were 
developed.  A  test  system  and  database  for  the  investigation  of  mechanisms  by  which  ligaments 
and  ligament  scars  'creep'  has  been  developed.  The  researchers  succeeded  in  altering  a 
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number  of  cells  in  ligament  scars  with  some  exciting  effects  on  ligament  organization  and 
mechanical  properties. 

The  number  of  supervised  trainees  included:  1  MSc/PDF  ('96  to  present),  3  additional  PDF's 
('96  to  present,  '97  to  present,  '97  to  '98)  and  one  Summer  Student  ('97). 

Interface  (b)  There  were  16  presentations  in  the  97/98  period.  (More  than  one  presentation 
was  given  at  each  conference).  They  included:  the  Combined  Meeting  of  the  Orthopaedic 
Association,  Keystone  Symposia,  Fifteenth  Annual  Surgeons  Day,  24*^  Annual  Alberta 
Orthopaedic  Resident  Research  Day,  Canadian  Orthopaedic  Association  and  the  31^^  Annual 
Meeting  of  the  Canadian  Orthopaedic  Research  Society,  and.  Transactions  of  the  43^^  Annual 
Meeting  of  the  Orthopaedic  Research  Society. 

Secondary  outputs  A  measuring  device  was  developed  in  Dr.  B's  lab.  Some  graduate 
students  took  this  development  one  step  further  and  formed  a  company  called  TENET.  They 
engineer  and  design  surgical  equipment  and  measuring  instruments.  Dr.  B  feels  that  this  kind 
of  technology  commercialization  may  be  a  way  of  funding  further  research.  Dr.  B  has  been 
cited  at  the  hand/joint  wound-healing  research  meetings,  probably  because  of  the  spill  over  of 
the  two  research  areas. 

Applications  This  research  will  not  go  forward  into  application  without  all  the  proper  testing. 
Dr.  B  estimated  it  would  be  5-10  years  before  gene  therapy  would  be  ready  for  trials  in  human 
patients. 

Impact/Final  Outcome  This  information  will  not  be  available  for  many  years.  Potential 
outcomes  would  include  better  patient  care,  possibly  a  safer  mechanism  for  ligament  repair. 
Cost  implications  have  not  been  addressed.  There  is  also  potential  for  commercial  interest  in 
this  particular  gene  therapy. 
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Case-Study  6:  Electrophysiology  of  the  heart 

Research  Needs  Assessment  Cardiac  arrhythmias  remain  a  major  source  of  morbidity  and 
mortality  in  western  societies.  A  thorough  understanding  of  the  membrane  currents  that  control 
excitation  forms  the  basis  for  an  approach  to  arrhythmia  prevention  and  treatment.  The  studies 
proposed  by  the  principal  investigator  would  contribute  a  significant  advancement  to  this  field. 
Dr.  C's  own  interest  in  this  work  began  as  a  student  more  than  20  years  ago  when  he  saw 
nerve/heart  cells  generate  impulses  spontaneously. 

From  a  clinical  point  of  view,  Dr.  C's  work  may  also  prove  to  be  relevant  in  understanding  some 
of  the  dysrhythmias  that  occur  associated  with  cardiac  inflammation.  His  work  in  characterizing 
mouse  heart  cell  electrophysiology  is  likely  to  come  into  its  own  as  increasing  numbers  of 
transgenic  animals  are  used  to  study  human  disease.  Dr.  C  uses  classical  electrophysiology  in 
combination  with  a  range  of  modern  techniques  to  understand  better  the  events  associated  with 
cardiac  pacemaker  activity  and  conduction.  As  one  reviewer  put  it,  supporting  the  downstream 
benefits  of  this  research:  "The  connection  between  understanding  SA  and  AV  node  function  and 
the  development  of  pharmacologic  interventions  is  obvious". 

The  principal  investigator  states  that  his  collaborative  work  is  in  part  'applied'  in  that  it  is  directed 
toward  developing  animal  models  of  human  disease  or  mathematical  models  of  human  atrium. 
Both  of  these  topics  are  relevant  to  forefront  clinical  research  throughout  North  America. 

Interface  As  is  normal  for  AHFMR  funded  research,  expert  scientific  review  results  in  the 
decision  to  fund  the  research.  If  the  reviewers'  comments  influenced  changes  in  the  research 
proposal  this  was  not  mentioned  when  Dr.  C  was  interviewed.  Comments  from  reviewers  were 
generally  positive  and  supported  the  research  plan. 

Inputs  Main  grant  support  comes  from  MRC  of  Canada  Group  Grant,  (1996  to  2001)  covering 
six  independent  projects  and  four  Core  Facilities.  In  addition,  support  is  provided  by  The  Heart 
&  Stroke  Foundation  of  Canada  (1995  to  1998),  an  endowed  Chair  sponsored  by  the  Alberta 
affiliate  of  The  Heart  &  Stroke  Foundation  of  Canada  (1995  to  2000,  renewable  to  2005, 
pending  review  in  1999).  The  Wellcome  Trust  has  given  a  Foreign  Starter  grant  over  5  years. 
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Dr.  C  continues  to  Head  a  Cardiac  Electrophysiology  laboratory.  His  education  includes  a  BSc 
and  MSc  (both  in  Physiology).  His  PhD  was  completed  at  Yale  University.  He  was  a  PDF  at 
Oxford  University.  A  second  PDF  was  completed  at  the  University  of  Washington  in 
Bioengineering.  He  was  Assistant  Professor  in  Physiology  at  the  University  of  Texas,  then 
Associate  Professor  of  Physiology  at  his  present  institution.  Currently  he  holds  the  position  of 
Professor  of  Physiology  and  Biophysics.  Dr.  C  was  awarded  an  American  Heart  Association 
Established  Investigators  Award  for  1980-83.  From  1995  to  the  present  he  has  had  the  Heart 
and  Stroke  Foundation  of  Alberta  Endowed  Research  Chair. 

There  is  an  ongoing  collaboration  and  consultation  with  the  pharmaceutical  industry.  Wyeth- 
Ayerst  has  provided  an  unrestricted  Educational  Grant  to  the  appropriate  core  of  our  MRC 
Group.  This  involves  free  exchange  of  information  mainly  concerning  priorities  and  directions 
for  development  of  genetically  altered  mouse  models  of  cardiovascular  disease.  A  second 
somewhat  similar  initiative  is  ongoing  with  Otsuka  Pharmaceuticals.  Within  a  month  additional 
meetings  will  be  held  with  Proctor  and  Gamble  Pharmaceuticals  and  one  other  international 
company. 

Processes  The  model  systems  used  to  characterize  membrane  currents  and  transporters  in 
cardiac  pacemakers  are  myocytes  isolated  from  sinoatrial  and  atrioventricular  nodes  and  the 
Purkinje  cells  of  the  conduction  systems.  Conventional  step  voltage  clamp  and  an  action 
potential  clamp  is  used  to  determine  the  time  and  voltage  dependence  of  the  membrane 
currents  that  control  repolarization  and  the  hyperpolarization-activated  current.  The  coupling 
between  pairs  of  cardiac  cells  and  the  conduction  between  groups  of  cardiac  cells  will  be  looked 
at  with  classical  voltage  clamp  studies  and  with  voltage  sensitive  dyes  in  a  collaborating 
laboratory.  The  membrane  currents  that  underlie  the  action  potential  in  the  mouse  heart  will  be 
studied  using  both  normal  and  transgenic  mice.  These  preparations  may  provide  useful  models 
of  a  number  of  human  diseases. 

Primary  outputs  There  were  1 1  publications  and  6  abstracts  that  came  out  of  this  research 
during  1996-1997.  The  journals  include  Journal  of  Physiology,  American  Journal  of  Physiology, 
Circulation  Research,  Canadian  Journal  of  Cardiology,  FEBS  Letters,  Chaos,  and  Nature. 

Key  findings  include  the  completion  of  a  mathematical  model  of  the  electrophysiological 
responses  in  human  atrium,  publication  of  the  first  full-length  description  of  the  mechanisms  of 
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repolarization  in  subsidiary  cardiac  pacemaker/conduction  tissue,  the  Purkinje  cell  from  rabbit 
heart.  Collaborative  work  done  with  scientists  at  the  University  of  Utah  resulted  in  improved 
understanding  of  electrontonic  coupling  between  cells  within  the  atrio-ventricular  node,  as  well 
as  myocytes  from  atrium  and  ventricle.  There  was  further  characterization  of  the  repolarization 
process  in  the  atria  and  ventricles  of  control  mouse  hearts,  and  hearts  rendered  arrhythmic  or 
exhibiting  congestive  failure  as  a  result  of  selected  genetic  engineering.  The  cloning  of  some  of 
the  potassium  channels  from  mouse  heart  has  been  carried  out  in  the  Molecular  Biology  and 
Educational  Cores  by  two  trainees.  The  PI  has  collaborated  with  a  researcher  at  the  University 
of  Alberta  who  is  an  expert  with  electronics  and  together  with  a  colleague's  computer  skills  they 
have  developed  a  high  speed,  high-resolution  ultrasensitive  camera  for  use  in  the  context  of 
studying  normal  cardiac  rhythms,  and  rhythm  disturbances. 

Dr.  C  supervises  postdoctoral  fellows  (4-5/yr)  and  PhD  candidates  1-2  at  his  University;  and  1-2 
at  Rice  University.  In  addition  he  sits  on  4  graduate  student  advisory  committees  and  does  3 
external  exams  per  year.  Teaching  duties  include  graduate  students,  the  nursing  class  and  the 
first  year  medical  class.  Dr.  C  is  participating  in  the  preparation  of  the  first  'electronic  textbook' 
for  physiology  and  biophysics  as  part  of  an  initiative  of  the  Biophysical  Society. 

Interface  (b)  Dr.  C  participated  in  the  establishment  of  a  lap  top  computer  familiarity/education 
program  at  a  local  Children's  Hospital.  As  well,  he  gave  public  presentations  for  the  Heart  and 
Stroke  Fund  during  annual  Heart  &  Stroke  Week.  Dr.  C  has  a  consistent  track  record  of 
productive  interactions  with  clinical  scientists.  This  has  involved  supporting  their  junior 
colleagues  (fellows)  and  writing  and  rewriting  their  applications  and  papers.  An  ongoing 
collaboration  exists  between  Dr.  C  and  Rice  University. 

In  July  of  1997  Dr.  C  and  his  colleagues  presented  research  at  the  International  Union  of 
Physiological  Sciences  in  St.  Petersburg,  Russia.  They  contributed  as  invitees  to  3  sessions. 

Secondary  outputs  The  genetically  altered  mice  that  Dr.  C  has  collaborated  in  designing,  with 
congestive  heart  failure  in  mind,  may  be  picked  up  by  the  pharmaceutical  industries.  The 
biotechnology  area  may  pick  up  the  use  of  the  camera,  designed  and  built  by  Dr.  Frank 
Witkowski  at  the  University  of  Alberta,  to  use  in  experiments  that  track  the  ventricular  rhythms, 
i.e.  the  genesis  of  fibrillation.  From  this  they  can  determine  where  to  concentrate  further 
research  areas.  As  well,  the  neurology  researchers  are  picking  up  the  tracking  dye/camera 
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ideas.  It  is  a  technology  that  works  even  better  in  brain  and  gut.  For  example,  the  high-speed 
camera,  with  a  tracking  dye,  may  show  how  a  drug  designed  for  gut  disorders  may  have  a 
direct,  negative  impact  on  the  heart.  The  mathematical  model  that  Dr.  C  has  created  is  the  first 
and  only  model  of  human  atrium.  Both  the  device  industry  and  the  pharmaceutical  industry  are 
interested  in  it. 

Applications  There  is  no  application  of  this  research  at  this  time. 

impact/Final  Outcome  As  there  has  been  no  application,  there  would  be  no  way  to  measure 
impact  or  final  outcomes. 
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Case-Study  7:  Lipid  metabolism 

Research  Needs  Assessment  Dr.  D  stated  that  the  impetus  to  start  him  on  this  course  of 
research  was  born  out  of  curiosity.  He  wanted  to  understand  lipid  metabolism  at  the 
biochemical  level  and  to  understand  the  basis  of  atherosclerosis.  Phospholipids  are 
components  of  all  cellular  membranes.  These  researchers  have  shown  that  biosynthesis  of  one 
of  these  phospholipids,  phosphatidylcholine  (PC),  is  necessary  for  cells  to  grow  and  divide.  The 
major  pathway  for  PC  biosynthesis  in  all  animal  cells  is  the  choline  pathway.  Liver  cells  are 
different  since  they  also  have  a  second  pathway,  the  PEMT  pathway.  It  was  not  obvious  why 
this  PEMT  pathway  exists  and  only  in  the  liver. 

Other  work  in  this  lab  includes  characterizing  the  enzyme,  triacylglycerol  hydrolase  (TGH).  It  is 
involved  in  the  secretion  of  fat  from  the  liver  into  the  blood  stream.  The  DNA  that  codes  for 
TGH  was  cloned  and  expressed  in  immortalized  liver  cells  in  culture.  The  cells  that  had  TGH 
used  fat  stored  in  these  cells  for  secretion  much  better  than  cells  that  did  not  have  TGH.  These 
and  other  results  suggest  that  TGH  may  have  an  important  role  in  regulating  the  mobilization 
and  secretion  of  fat  from  liver.  The  enzyme  is  a  potential  target  for  therapeutic  intervention  to 
lower  serum  fats  and  other  lipids. 

In  the  past  research  also  concentrated  on  investigating  the  role  of  apolipoprotein  E  in  neurons  in 
relation  to  the  putative  linkage  between  an  accumulation  of  an  isoform  of  apo  E  and  Alzheimer's 
disease.  Since  neurons  do  not  make  apo  E,  a  fundamental  question  is:  how  does  apo  E  enter 
neurons  and  escape  proteolytic  degradation?  The  fate  of  cholesterol  and  phospholipids  that  are 
associated  with  apo  E  when  added  to  the  medium  of  cultured  neurons  will  be  characterized. 

Interface  As  is  normal  for  AHFMR-funded  research,  expert  scientific  review  informs  the 
decision  to  fund  the  research.  If  the  reviewers'  comments  influenced  changes  in  the  research 
proposal  this  was  not  mentioned  when  Dr.  D  was  interviewed.  Comments  from  reviewers  were 
generally  positive  and  supported  the  research  plan.  Dr.  D  mentioned  that  his  research  is  strictly 
investigator-driven  with  the  MRC  and  the  Heart  Foundation.  The  area  of  lipid  metabolism  is 
extremely  competitive.  There  are  no  major  collaborations  outside  the  University.  There  are  4 
different  faculty  members  that  interact  in  planning  future  research  goals. 
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Inputs  Dr.  D  holds  a  'Scientist'  award  from  the  AHFMR.  In  addition  he  has  an  operating  grant 
from  the  Heart  &  Stroke  Foundation  of  Canada,  a  shared  operating  grant  from  the  Medical 
Research  Council  of  Canada,  an  operating  grant  from  Medical  Research  Council  of  Canada, 
and  an  operating/contract  grant  from  Glaxo-Wellcome.  The  names  of  the  projects  for  each  of 
these  grants  are:  Phosphatidylcholine  and  lipoprotein  metabolism,  Metabolism  and  trafficking  of 
apoE  and  lipids  in  neurons.  Regulation  of  phosphatidylcholine  metabolism,  and  Triacylglycerol 
hydrolase,  respectively. 

Dr.  D  obtained  his  BSc  (Chemistry)  from  Dickinson  College,  his  PhD  in  biochemistry  from  the 
University  of  Pittsburgh.  He  did  a  PDF  at  the  University  of  Pittsburgh,  a  PDF  at  Harvard 
University  and  a  final  PDF  at  the  University  of  Warwick.  He  began  as  Assistant  Professor  at  a 
Canadian  Institution  and  left  there  a  full  Professor.  He  also  held  the  positions  of  Associate  Dean 
of  Medicine  and  then  Head  of  the  Department  of  Biochemistry.  He  started  as  a  Professor  of 
Biochemistry  at  his  current  university  in  1986.  He  is  also  the  Director  of  the  Lipid  and 
Lipoprotein  Research  Group.  Dr.  D  is  a  Scientific  Officer  for  the  Medical  Research  Council,  a 
reviewer  for  Heart  and  Stroke  Foundation  of  Canada,  a  reviewer  for  the  Medical  Research 
Council  of  Canada,  a  reviewer  for  the  Biochemical  Journal  Advisory  Board,  a  reviewer  for 
Progress  in  Lipid  Research,  a  reviewer  for  Journal  of  Lipid  Research  Advisory  Board  and  an 
Executive  Editor  for  Biochimica  Biophysica  Acta. 

Processes  It  was  not  obvious  why  the  PEMT  pathway  exists  and  only  in  the  liver.  The  gene 
that  codes  for  PEMT  was  isolated,  altered  genetically  in  a  test  tube  then  cloned  into  mice.  With 
this  modified  gene,  mice  were  produced  in  which  PEMT  was  no  longer  present.  Further 
experiments  were  done  to  determine  the  function  of  PEMT.  Other  work  in  this  lab  included 
characterizing  the  enzyme,  triacylglycerol  hydrolase  (TGH).  It  is  involved  in  the  secretion  of  fat 
from  the  liver  into  the  blood  stream.  The  DNA  that  codes  for  TGH  was  cloned  and  expressed  in 
immortalized  liver  cells  in  culture.  The  cells  that  had  TGH  used  fat  stored  in  these  cells  for 
secretion  much  better  than  cells  that  did  not  have  TGH.  These  and  other  results  suggest  that 
TGH  may  have  an  important  role  in  regulating  the  mobilization  and  secretion  of  fat  from  liver. 
The  enzyme  is  a  potential  target  for  therapeutic  intervention  to  lower  serum  fats  and  other  lipids. 

In  the  past  research  also  concentrated  on  investigating  the  role  of  apolipoprotein  E  in  neurons  in 
relation  to  the  putative  linkage  between  an  accumulation  of  an  isoform  of  apo  E  and  Alzheimer's 
disease.  Since  neurons  do  not  make  apo  E,  a  fundamental  question  is:  how  does  apo  E  enter 
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neurons  and  escape  proteolytic  degradation?  The  fate  of  cholesterol  and  phospholipids  that  are 
associated  with  apo  E  when  added  to  the  medium  of  cultured  neurons  will  be  characterized. 

Primary  outputs  In  1997,  Dr.  D  has  had  8  publications  and  5  book  chapters.  The  journals  are: 
Biochemistry  Journal,  Journal  of  Biological  Chemistry,  Journal  of  Lipid  Research,  Biochimica 
Biophysica  Acta,  and  Proceedings  of  the  National  Academy  of  Science.  He  currently  has  4 
graduate  students  and  3  PDFs. 

Interface  (b)  Dr.  D  has  participated  in  scientific  conferences  of  the  Dept  of  Biochemistry  and 
Molecular  Biology,  Uniformed  Services  University,  Washington,  D.C.;  International  Conference 
on  Biochemistry  of  Lipids,  Italy;  Kern  Aspen  Lipid  Conference,  Colorado;  Dept  of  Veterinary 
Biochemistry,  Utrecht  University,  The  Netherlands;  Dept  of  Biological  Sciences,  Carnegie- 
Mellon  University  of  Pittsburgh;  Glaxo-Wellcome  Research  Centre,  Paris;  Dept  of  Biochemistry 
and  Molecular  Biology,  University  of  Manitoba. 

Secondary  outputs  Other  labs  have  made  use  of  the  PEMT  genetically  altered  mice.  There 
was  an  application  of  Dr.  D's  and  other's  earlier  research  that  resulted  in  a  phospholipid  being 
developed  that  is  extremely  important  in  lung  surfactant/infant  respiratory  distress  syndrome. 

Applications  There  have  been  no  applications  of  Dr.  D's  current  research  at  this  time. 

Impact/Final  Outcome  Without  application  of  the  research  the  impact  and  final  outcomes 
cannot  be  measured. 
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Conclusions  from  the  Case  Studies 

In  all  four  cases  the  research  sequence  model  seemed  to  provide  a  useful  framework  for 
reviewing,  ex  post,  what  had  been  the  benefits  from  the  research.  There  seemed  a  priori  Xo  be 
particular  merit  in  focusing  on  the  two  interfaces,  and  analysis  of  these  identified  some 
interesting  issues. 

None  of  these  streams  of  research  had  in  any  sense  been  commissioned:  there  were  no  formal 
prior  processes  of  needs  assessment  by  the  AHFMR  or  other  independent  parties.  It  was, 
therefore,  interesting  to  see  how  the  researchers  had  identified  the  original  topics  and  how  they 
continued  to  identify  key  research  sub-topics  and  objectives  at  each  stage  of  their  work.  What 
emerged  was  that,  typically,  the  original  interest  in  the  body  of  research,  the  broad  topic  or 
theme,  arose  through  happenstance.  For  example,  one  of  the  clinical  researchers  whole 
interest  had  been  triggered  by  a  personal  medical  problem:  one  of  the  scientists  had  been 
stimulated  by  one  particular  lecture  attended  as  a  student.  Subsequently,  their  interests  and 
focus  may  have  moved  to  being  more  or  less  curiosity-driven  or  problem-led.  It  appeared  that 
Dr.  A  had  been  initially,  and  remained,  essentially  problem-led.  His  aim  was  to  apply  intellectual 
rigour  to  finding  a  better  clinical  therapy  for  diabetic  patients.  Dr.  D  had  been  and  remained 
curiosity  driven:  his  intellectual  challenge  was  to  achieve  a  better  understanding  of  lipid 
metabolism  at  the  molecular  level.  Dr.  C  had  initially  been  entirely  curiosity  driven,  but 
increasingly  recognized  or  accepted  the  importance  of  ensuring  the  clinical  relevance  of  his 
work.  Dr.  B's  interest  had  been  initiated  from  an  applied  concern  to  improve  on  available 
therapy  but  increasingly  he  saw  the  need  to  go  back  to  very  basic  science  to  begin  to  provide  a 
scientific  underpinning,  and  was  increasingly  fascinated  by  that  basic  science. 

While  this  no  doubt  represents  both  an  oversimplification  and  perhaps  an  over-reading  of  brief 
informal  interviews,  it  raises  some  interesting  questions.  Why  have  some  of  these  researchers 
moved  their  original  curiosity-driven,  or  health  needs-led  positions?  For  the  clinicians,  moving 
towards  basic  science,  it  may  on  the  one  hand  be  recognition  of  the  need  to  understand 
mechanisms  by  trying  to  determine  the  molecular  basis  of  a  disease  state  before  you  can 
properly  treat  it.  It  may  also  be  the  result  of  experience  that  has  found  bench  science  easier  to 
deal  with  than  persuading  subjects  and  clinical  colleagues  to  participate  in  clinical  trials.  For 
some  basic  scientists  the  need  to  focus  on  clinical  application  may  reflect  not  only  a  real  interest 
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in  improving  healtii,  but  also  a  recognition  of  the  access  it  provides  to  alternative  sources  of 
funding. 

Overall  it  suggests  that  the  differences  between  basic  scientists  and  clinical  investigators  may 
not  be  as  stark  as  might  be  expected,  and  that  as  careers  develop  the  balance  of  motivation 
may  change. 

The  subsequent  process  of  defining  and  refining  research  strategy  and  specific  research 
objectives  also  differed.  The  relative  importance  of  personal  views  on  direction,  of  the  ideas  of 
an  immediate  group  of  colleagues  within  the  university  and  of  influences  from  a  wider 
international  research  community  varied.  In  some  cases,  the  strategy  was  a  personal  one, 
devised  in  the  context  of  what  was  seen  as  an  essentially  competitive  international  research 
community.  In  other  cases  what  emerged  were  quite  significant  attempts  to  define  an  agenda 
with,  and  to  some  extent  to  agree  to  a  division  of  the  research  amongst,  groups  of  researchers 
working  internationally  on  a  basis  of  friendly  cooperative  rivalry. 

In  none  of  the  cases  studied  did  there  seem  to  have  been  a  significant  interaction  between  the 
researcher  and  the  AHFMR  (in  terms  of  its  staff  or  its  scientific  advisers  and  reviewers)  in 
refining  or  focusing  a  research  program  or  research  strategy.  The  researchers  welcomed  this 
lack  of  interference,  but  it  may  represent  a  lost  opportunity  to  get  the  greatest  benefit  from  the 
AHFMR's  international  scientific  review  process. 

In  considering  inputs,  what  emerged  unprompted  was  recognition  of  the  very  privileged 
circumstances  they  have  enjoyed  through  the  long-term  support  of  the  AHFMR.  All  noted  that 
this  was  a  major  advantage  to  working  in  Alberta,  and  one  that  gave  them  a  competitive 
advantage  over  colleagues  elsewhere. 

The  second  interface  in  the  model  relates  to  the  way  in  which  results  are  disseminated,  or  fed 
into  the  international  reservoir  of  knowledge  ready  to  be  picked  up  and  used  by  others.  One 
point  of  important  difference  that  showed  itself  in  the  interviews  was  the  weight  different 
individuals  placed  on  publications  as  a  measure  of  research  output  and  as  the  main  means  of 
communicating  or  disseminating  results.  For  some,  publications  were  the  key  measure  of 
success  and  the  key  means  of  communication.  For  others,  other  indicators  of  success  were  as 
important,  for  example  influencing  colleagues  clinical  behaviour  and  a  key  means  of 
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dissemination  was  through  more  informal  communications  within  research  'networks'  that 
brought  together  the  key  research  players  working  on  a  particular  topic.  Tentatively,  it  seems  to 
be  the  case  that  there  is  a  relationship  between  degree  of  concern  about  the  extent  to  which 
research  results  influence  others,  particularly  in  terms  of  the  downstream  application  of  research 
results,  and  the  perceived  importance  of  outputs  other  than  publications.  This,  in  turn,  suggests 
that  the  value  and  relevance  of  bibliometric  measures  may  vary  between  areas.  Certainly  the 
differences  in  opinion  expressed  in  this  study  suggest  that  such  measures  cannot  be  ignored 
but  should  not  be  used  alone,  incautiously  or  crudely.  Further  evidence  is  needed  on  the 
correlation  between  bibliometric  measures  and  other  more  subjective  measures  of  research 
'value'  and  impact  (40). 

Until  now,  accounts  of  the  Buxton/Hanney  model  have  stressed  that  research,  even  at  the 
applied  end  of  the  spectrum,  feeds  back  into  future  research.  Research  paybacks  are  the 
second  main  Buxton/Hanney  category  of  payback.  From  the  earliest  account  of  the  linear 
model,  it  was  recognized  that  feedback  loops  existed  from  the  primary  output  of  research  back 
to  further  research.  Subsequent  iterations  of  the  model  have  emphasized  the  complexity  and 
importance  of  such  feedback  loops,  but  have  never  attempted  to  analyze  them  specifically.  If 
the  model  is  to  be  used  for  basic  research  then  it  would  seem  important  to  begin  to 
conceptualize  how  these  feedback  loops  operate,  and  to  categorize  what  is  being  fed  back. 

This  is  a  step  that  needs  more  analysis  but  at  first  sight  the  following  categories  emerge: 

•  specific  pieces  of  knowiedge  that  fill  gaps  in  understanding  and  so  enable  a  further 
line  of  research  to  be  pursued 

•  development  of  broad  research  models,  or  techniques  or  methods 

•  development  of  possible  therapeutic  interventions  that  need  proper  evaluation  and 
testing 

•  transfer  of  ideas  from  one  therapeutic  area,  or  organ,  or  disease  system,  to  another 
outside  the  range  of  the  original  research. 

Alternatively,  it  might  be  conceptualized  in  terms  of  two  types  of  influence  that  research  might 
have  on  subsequent  research.  It  may  produce  incremental  knowledge:  bit  by  bit  filling 
knowledge  gaps  or  providing  building  blocks  for  subsequent  research,  or  identifying  dead  ends. 
Alternatively,  it  may  produce  a  dramatic  conceptual  change:  a  breakthrough  in  how  a 
problematic  issue  is  perceived,  refuting  or  challenging  previous  research  or  current  perceptions 
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and  requiring  a  major  readjustment  of  how  to  proceed.  It  has  been  argued  more  generally  that 
the  "impact  of  research  information  depends  upon  its  ability  to  change  beliefs  or  assumptions 
within  the  relevant  policy  audience"  (28). 

Additionally,  the  research  process  itself,  rather  than  its  specific  outcomes,  may  feed  into  future 
research  through  the  development  of  trained  personnel  or  tools  and  techniques  that  can  be 
used  by  others  that  were  not  the  formal  research  objective. 

It  would  appear  that  this  aspect  of  the  model  -  research  feedbacks  -  could  beneficially  be 
developed,  focusing  logically  on  the  'dissemination'  interface  through  which  these  feedbacks 
occur,  particularly  when  the  feedback  is  to  research  being  undertaken  by  quite  separate  groups. 
For  example,  in  analyzing  publications  and  dissemination  via  conferences  etc,  it  may  be  useful 
to  try  to  distinguish  the  types  of  audience  to  which  they  are  directed.  The  type  of  journal  may  be 
a  proxy  indicator  for  the  type  of  audience:  generalist,  specialist  or  sub-specialist.  In  order  for 
research  ideas  to  migrate  from  the  basic  science  bench  to  application  in  the  clinic,  it  may  need 
to  be  disseminated  out  of  the  narrow  specialty  into  the  thinking  of  a  wider  scientific  audience. 
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SECTION  6; 
THE  NEXT  STEPS 


The  aims  of  this  project  have  been  essentially  methodological:  to  test  the  Buxton/Hanney 
approach  in  the  context  of  the  work  of  the  AHFMR.  In  the  short  period  involved  some  of  the 
conclusions  from  this  are  necessarily  somewhat  tentative,  and  need  further  confirmatory  testing. 
This  final  section  attempts  to  summarize  what  has  been  found  and  to  propose  a  series  of 
possible  next  steps.  These  fall  into  three  broad  categories: 

•  Steps  that  the  AHFMR  might  reasonably  take  to  apply  this  approach  as  it  stands; 

•  Further  work  that  the  AHFMR  might  undertake  or  commission  to  further  develop  and  test 
more  formally  approaches  for  its  future  use; 

•  A  number  of  key  issues  that  need  to  form  part  of  a  broader  research  agenda  in  which  the 
AHFMR  might  wish  to  be  involved  but  which  is  not  specific  to  it. 

In  each  case  the  suggestions  or  recommendations  are  offered  here  for  discussion  and  would 
benefit  from  further  thought,  development  and  refinement. 

immediate  use  by  the  AHFMR 

It  would  appear  that  the  approach  could  readily  be  used  for  all,  or  a  larger  sample  of  the  projects 
undertaken  with  the  HTA  Unit's  program  of  work.  The  objectives  of  this  would  be  two-fold: 

•  to  provide  a  monitoring  device  and  a  process  of  accountability,  hopefully,  to  justify  to 
Alberta  Health  the  continuation  and  possible  expansion  of  this  program; 

•  to  begin  to  indicate  which  circumstances,  or  types  of  projects,  have  the  greater  or  clearer 
'payback',  as  an  aid  to  future  prioritization  and  to  find  ways  of  improving  future  payback. 

A  number  of  in  depth  assessments  of  payback  from  AHFMR-funded  HSR  could  be  undertaken, 
to  serve  as  illustrative  examples  to  use  as  evidence  of  the  value  of  this  type  of  work.  These 
case  studies  would  be  more  formalized  versions  of  the  types  of  analyses  already  undertaken  for 
the  AHFMR  (35).  Those  presented  here  could  be  used  but  they  would  probably  require  further 
work  to  complete  and  validate  them.  The  AHFMR  might  undertake  some  formal  or  informal 
experiments  to  structure  the  advice  and  'steering'  provided  to  some  projects  with  an  aim  to 
maximizing  potential  payback.  It  might,  for  example,  help  researchers  obtain  the  'buy  in'  to  the 
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research  of  key  individuals  eventually  likely  to  be  concerned  with  policy  development  from  the 
research. 

it  would  be  useful  to  review  the  routine  data  collected  on  HSR  projects,  to  see  whether  this 
could  be  adapted  better  to  provide  indications  of  payback.  For  example,  the  following  might  be 
helpful: 

•  applicants  might  be  asked  to  more  clearly  structure  parts  of  the  application  to  set  out 
how  the  basis  of  their  needs  assessment  for  the  project,  the  form  of  the  interfaces,  how 
they  see  secondary  outputs  arising,  and  what  they  can  do  to  encourage  these; 

•  scientific  reviewers  might  also  comment  on  these  aspects; 

•  some  system  of  post  project  end  monitoring,  for  example  a  report  two  years  after  the 
end  of  the  project  might  be  required. 

The  AHFMR  already  undertakes  a  large  amount  of  general  dissemination  about  projects. 
However,  it  might  wish  to  see  whether  it  could  actively  focus  on  attempting  to  ensure  that 
appropriate  secondary  outputs  are  achieved.  For  instance,  it  might  bring  together  professional 
leaders  to  develop  and  endorse  guidelines  reflecting  and  encompassing  specific  research 
results,  or  support  other  fora  at  which  research-based  health  policies  might  be  developed. 

While  firm  recommendations  relating  to  basic  science  research  need  for  the  most  part  to  await 
further  developmental  work  (such  as  that  set  out  in  the  next  sub-section),  it  might  nevertheless 
be  worth  considering  whether  the  full  potential  of  'interface  (a)'  could  be  realized.  This  could  be 
accomplished  by  experimenting  with  a  process  of  feeding  back  to  applicants  the  reviewers' 
comments  on  their  applications  and  encouraging  them  to  provide  their  response  to  these 
comments  prior  to  their  proposal  being  considered.  This  approach,  used  by  the  UK  MRC, 
actively  encourages  applicants  to  take  on  board,  or  present  an  argument  to  counter, 
suggestions  made  about  focusing  research,  about  improving  methods,  etc.  At  present,  it 
appears  that  these  comments  are  only  provided  'for  information'  after  a  funding  decision  has 
been  made  and  when  there  may  be  little  incentive  to  consider  changes. 

Further  developmental  work  by  the  AHFMR 

It  has  not  been  feasible  to  undertake  detailed  citations  analysis  alongside  the  clinical/basic 
science  case  studies.  It  would  be  very  valuable,  using  citations  analysis,  to  try  to  trace  and  map 
the  pathways  through  which  some  of  the  research  results  have  been  taken  up  in  other  research. 
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It  may  be  possible  to  trace  these  ideas  through  citations  to  specific  applications,  for  which 
associated  health  benefits  might  be  assessed.  This  analysis,  if  it  is  to  involve  review  of  citations 
(not  simply  counting  them),  will  be  time-consuming  and  would  probably  benefit  from  the  skills  of 
an  experienced  librarian  familiar  with  the  ISI  database  and  someone  reasonably  familiar  with  the 
substantive  topic. 

Such  work  might  then  form  the  basis  for  beginning  to  form  a  view  as  to  whether  the  AHFMR 
should  invest  further  in  using  citations  analysis  to  better  understand  the  significance  of  the 
publication  outputs  from  projects.  Such  an  initiative  might  possibly  be  made  in  collaboration 
with  other  Canadian  funding  agencies,  and  (or)  the  Universities  of  Alberta  and  Calgary. 

The  conclusions  and  suggestions  relating  to  the  use  of  the  model  for  analyzing  clinical  and 
basic  science  are  necessarily  still  quite  tentative  at  this  stage.  If,  nevertheless,  they  are  seen  as 
promising,  then  some  additional  case  studies  should  be  undertaken. 

It  would  be  useful  to  find  a  way  of  exploring  some  projects  that  have  been,  or  are  perceived  as 
being  relative  failures,  to  see  whether  this  analytical  approach  can  help  identify  whether  they 
have  indeed  generated  little  payback,  and  if  so  why  that  may  have  been.  In  particular,  with 
hindsight,  could  anything  have  been  done  that  might  have  made  them  more  successful? 

The  AHFMR  might  wish  to  set  up  a  seminar  to  advertise  its  interest,  to  encourage  a  wider  local 
debate  about  these  issues,  and  to  obtain  additional  views  as  to  how  best  to  take  the  issues 
forward.  In  particular,  it  would  be  good  to  involve  some  of  the  basic  scientists  who  perhaps  see 
this  approach  as  rather  threatening  to  curiosity-led  research. 

The  broader  research  issues 

It  remains  unclear  as  to  how  different  are  the  results  from  bibliometric  approaches  and  these 
more  qualitative  approaches  such  as  that  of  Buxton/Hanney,  and  how  the  two  approaches  can 
most  usefully  complement  each  other.  Other  work  being  undertaken  at  present  by  Buxton  and 
Hanney  for  North  Thames  NHS  Executive  R&D  Program,  and  similar  studies  in  one  other  UK 
Region,  may  shed  some  light  on  this  but  further  systematic  work  is  needed. 

There  is  clearly  an  important  sub-text  running  through  this  work  as  to  whether  more  focusing, 
priority  setting,  mission-orientation  (or  whatever)  by  the  AHFMR  would  increase  or  reduce  the 
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payback  from  research.  There  appears  to  be  very  little  comparative  analysis  of  this.  Currently 
the  AHFMR  represents  the  case  of  non-direction.  Are  there  other  health  research  funding 
programs  in  Canada  that  reflect  a  more  directed  approach  and  could  be  analyzed 
comparatively?  (The  Quebec  HTA  program  may  be  an  example  (22-25)). 

These  proposals  do  not,  of  course,  constitute  a  complete  agenda  for  possible  work  on 
'payback'.  This  report  has  not  formally  considered  the  value  to  the  AHFMR  of  using  analysis  of 
potential  payback  to  inform  itself  of  the  possible  magnitudes  of  benefit  that  might  be  achieved 
from  specific  projects  (27,36).  It  may  be  even  more  ambitious  to  use  such  information  in  the 
process  of  prioritizing  research  spending  (15).  Such  approaches  have  been  considered  in  the 
USA  by  the  Institute  of  Medicine  (16,21)  and  are  again  under  serious  consideration  in  the  UK 
(20).  While  there  is  evidence  that  they  can  inform  individual  decisions,  particularly  those 
involving  major  research  expenditure,  as  yet  their  value  In  prioritization  is  largely  unproven. 

With  such  ex  ante  studies,  as  with  the  ex  post  analysis  on  which  this  report  focuses  it  is  clear 
that  there  is  a  growing  international  interest.  What  is  needed  is  a  vigorous  international  debate 
about  the  best  methods  for  ensuring  accountability  of  research,  for  ensuring  value  for  money 
from  research  programs,  and  for  deciding  whether  or  not,  and,  if  so,  how,  to  prioritize  research, 
applied  or  basic.  The  issues  will  not  go  away;  they  have  been  around  largely  unresolved  at 
least  since  the  1960's.  In  the  absence  of  approaches  that  are  tested,  evidence-based  and 
acceptable  to  the  broad  research  community,  political  and  bureaucratic  pressures  will  see  the 
introduction  of  methods  that  have  not  been  tested,  are  not  evidence-based  and  may  well  not 
command  the  support  of  the  research  community.  The  AHFMR  is  in  a  strong  position  to  take  a 
lead  role  in  pursuing  this  important  international  debate. 
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